Shadin A Garbage in, garbage out
“AI is only as good as the data you give it.” In machine learning, this idea is often summarized as “Garbage In, Garbage Out.” No matter how advanced a system becomes, the quality of its output depends mainly on the quality of its input. This matters most in finance, where decisions based on flawed data can lead to high financial risk.
Nowadays the internet is filled with buzzwords. XAI, AGI, ASI and world models are everywhere right now, and fintech sits right in the middle of that conversation. If we set the hype aside, they all point to one simple thing. Machines are becoming increasingly good at recognizing patterns, predicting outcomes, and assisting people in making complex decisions. Finance has become one of the main places where these capabilities are being tested in real time.
Financial transactions today are faster and more convenient than ever. Not long ago, many processes depended on middlemen, manual verification, paperwork and long approval cycles. Much of that friction is slowly being reduced as intelligent systems take over repetitive and time-consuming steps. The transition towards digital finance is largely possible because of artificial intelligence.
How it started
A major turning point came when Google released the research paper “Attention Is All You Need.” This paper introduced the transformer architecture, which later became the recipe for many frontier models we use today, such as GPT, Claude, and Gemini. Since many companies are building on this same underlying approach, modern systems often share similar issues and constraints.
One of the biggest challenges is trust. Many advanced AI systems still operate like a black box. They produce answers, but the reason behind those outputs is not always transparent to users. In finance, where accountability and auditability are essential, this lack of clarity is a serious concern.
Current LLMs are stateless. They are not good at carrying continuous awareness or having a lived experience from one moment to the next. They produce answers based on the user’s prompt and the patterns they have learned during training. This is also why AI does not possess goals or intentions of its own. At its core, it predicts the next most likely token based on probability rather than pursuing any personal objective.
AI is designed to be helpful and to satisfy the user. Because of this, instead of admitting uncertainty, an AI may sometimes produce something plausible rather than clearly stating a lack of knowledge. In many cases, it may appear to agree with the user’s assumption. This trait of AI is known as sycophancy.
There is also a common misunderstanding around “AI thinking.” When an AI shows its reasoning in English, it is usually a made-up explanation for human readability, because the system does not literally think in English. Internally, it operates through mathematical calculations, similarity searches and statistics.
Just like with humans, what an AI thinks can be totally different from what it says; its outputs depend on deeper internal processes. Today’s AI lacks embodiment or grounding. It works inside a story, almost like functioning within a dream built from text, numbers, and patterns. Without direct interaction with reality, its understanding can remain abstract.
Humans show a similar behaviour while dreaming. Inside a dream, people often accept absurd situations without questioning the story. Large language models can behave in a comparable way. Once they are inside a generated narrative, they tend to continue it smoothly rather than stepping outside and questioning the assumptions.
The concept of Artificial General Intelligence is quite far from reality, mainly because of these limitations. At present, systems still have fundamental problems to solve in terms of reliability, grounding, and usability in real-life situations.
AI agents are also one of the major growing trends. These are systems capable of multi-step tasks such as searching, comparing options, placing orders, and managing schedules. Peter Steinberger, the creator of OpenClaw, recently suggested that as many as 80% of today’s standalone apps could fade away as intelligent agents begin handling everyday tasks such as finding products, placing orders, managing routines, and assisting with fitness or diet decisions.
At the same time, industries are increasingly turning toward finance as the next major space for AI-driven transformation. Fintech can already perform compliance checks automatically, detect fraud at an earlier stage, offer tailor-made advice, and facilitate cross-border payments. The transformation is underway.
Today, manipulating an AI is surprisingly easy. Attacks such as “indirect prompt injection”, which tamper with an AI’s context or its chain of thought, show that an AI system’s behaviour can be steered through cleverly crafted inputs. Hence, security, transparency and good governance become crucial if the industry is to move toward a more profound level of automation.
The future of AI in fintech will depend on whether the industry can earn trust. There is no doubt that AI is powerful; what remains is a matter of trust and reliance.