AI Agents: A Revolution in How We Interact with Tech

Updated: April 30 2024 02:24

In recent years, the rapid advancements in artificial intelligence (AI) have led to the emergence of AI agents, which are revolutionizing the way businesses interact with customers and how people access the internet. From Bland AI's billboard advertising its AI agent that can handle phone calls for businesses in any voice (see below video) to the development of AI agents like Devin and Devika, the future of AI appears to be increasingly agentic. This post explores the rise of AI agents, their potential impact on various industries, and the opinions of experts in the field.

My next post is going to talk specifically about AI personal assistants (or some call it "AI Phone") where LLMs will be able to handle a wide range of intelligent tasks seamlessly and autonomously for consumers.

This is one cool billboard advertising promoting AI agent. Calling that number will connect u to a live conversation with an AI bot powered by @usebland. pic.twitter.com/A9tCLFU5dP

— Alvin Foo (@alvinfoo) April 25, 2024

Andrew Ng, founder of DeepLearning.AI and AI Fund, highlights the transition from non-agentic workflows, where users simply type a prompt and receive an answer from a language model (LLM), to agentic workflows. In an agentic workflow, the LLM engages in an iterative process, such as writing an essay outline, conducting research, drafting, analyzing, and revising the content. This approach has been shown to deliver remarkably better results compared to zero-shot prompting.


The benefits of agentic workflows are significant. Andrew Ng's team at AI Fund found that when GPT-3.5 was wrapped in an agentic loop, its performance on the HumanEval coding benchmark improved from 48.1% to an impressive 95.1%. This highlights the power of allowing AI agents to engage in self-reflection, planning, and multi-agent collaboration.




Ng highlighted the power of reflection in AI agents. For instance, if we ask a system to write code for a given task, we can then present the same code back to the LLMs and instruct it to carefully check the code for correctness, efficiency, and good structure. The same LLMs that wrote the code might spot bugs and suggest fixes. This method often works well and is worth trying in many applications. Ng recommended the following papers to learn more about Reflection:

• Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023)


• Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023)


• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024)

The use of tools has broadened the application range of LLMs. Many of us are familiar with how LLM-based systems use various tools to enhance functionality. For example, when you ask an LLMs something like "What's the best coffee machine on the internet?" or generate and execute code, it indicates that many different tools are widely used for analysis, information gathering, taking action, and boosting personal productivity. Ng recommended the following papers to learn more about Tool Use:

• Gorilla: Large Language Model Connected with Massive APIs, by Patil et al. (2023)


• MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action, by Yang et al. (2023)


• Efficient Tool Use with Chain-of-Abstraction Reasoning, by Gao et al. (2024)

Planning and multi-agent collaboration are emerging fields. In agentic workflows, the AI agent is capable of developing and executing multi-step plans to achieve specific goals.For example, if asked to conduct online research on a topic, the agent can break down the objective into smaller subtasks, such as researching specific subtopics, synthesizing findings, and compiling a report. Ng recommended the following papers to learn more about Planning:

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, by Wei et al. (2022)


• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, by Shen et al. (2023)


• Understanding the planning of LLM agents: A survey, by Huang et al. (2024)

Agentic workflows can involve multiple AI agents working together, each playing a different role, to arrive at better solutions than a single agent could. When using these technologies, their performance sometimes surprises us, but at least for now, their reliability still needs to be improved. Ng shared an example where two AI agents - one acting as a coder and the other as a reviewer - can be paired. This approach greatly enhances the performance of LLMs. Ng recommended the following papers to learn more about Multiagent Collaboration:

• Communicative Agents for Software Development, by Qian et al. (2023)


• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, by Wu et al. (2023)


• MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework, by Hong et al. (2023)

Venture capitalist Vinod Khosla envisions a future where most consumer access to the internet will be through agents acting on behalf of consumers, performing tasks and fending off marketers and bots. He predicts that tens of billions of agents on the internet will become the norm. Similarly, Meta CEO Mark Zuckerberg believes that the future of AI lies in creating agents that can accomplish complex tasks and represent the interests of businesses and creators. He sees the potential for creators to own and train AI agents to engage with their communities, overcoming the limitations of time and resources.


Tech giants and startups alike are investing heavily in the development of AI agent platforms. Google has introduced Vertex AI Agent Builder, which enables the creation of autonomous agents with minimal coding. It can help design, deploy, and manage AI agents for conversation and process automation using natural language. It offers prompt-based tools and pre-built templates for quick prototyping and deployment. The system provides real-time testing, monitoring, and performance adjustment of the agents.

NVIDIA has partnered with Hippocratic AI to develop GenAI agents that outperform human nurses on video calls at a lower cost. This partnership is focused on transforming global healthcare accessibility and outcomes by leveraging deep healthcare expertise. The goal is to enhance patient interactions by making them faster, smoother, and more personalized. The collaboration involves NVIDIA's Avatar Cloud Engine and Riva models, which provide automatic speech recognition and text-to-speech translation. Combined with Hippocratic AI's safety approach and large-scale deployment capabilities, it aim to establish a new benchmark for empathetic AI healthcare agents.

Amazon launched Amazon Q, a generative AI-powered assistant built on Amazon Bedrock. It provides contextual guidance, explains program logic, identifies and fixes bugs, and even implements complete features along with test cases. Agents for Amazon Bedrock, the power behind Amazon Q, enable generative AI applications to execute multistep tasks across company systems and data sources. Creating an application with Amazon Q and Agents for Amazon Bedrock involves naming the application, selecting retriever, configuring data sources, adding plugins, configuring global controls, and defining topic guardrails. With these AI Agents tools, developers can focus more on innovation and less on mundane tasks.


Startup's AI agents like Cognition AI's Devin Software Engineer has demonstrated their potential in real-world scenarios. Devin has successfully resolved nearly 14 out of every 100 software development issues, showcasing AI's ability to autonomously understand and address complex problems. Devika, an Indian open-source AI software engineer, can understand human instructions, break them down into tasks, conduct research, and write code to achieve set objectives. These advancements highlight the growing capabilities of AI agents and their potential to support various industries.


Devika aims to revolutionize the way we build software by providing an AI pair programmer who can take on complex coding tasks with minimal human guidance. Check out their work at their Devika open source Github repo.


Another popular open source software development agent is called ChatDev. It is designed to revolutionize software development by leveraging the power of Large Language Models (LLMs). Imagine a virtual software company staffed by various AI agents - programmers, reviewers, testers, and even designers. ChatDev utilizes these agents to collaborate through a series of focused chats, tackling each stage of development from requirement analysis to final documentation.

This "chat chain" approach ensures meticulous attention to detail and minimizes errors by fostering communication and cross-examination between agents. By automating tasks and streamlining the process, ChatDev offers a faster, more cost-effective, and potentially more reliable approach to building software. More details in their paper Communicative Agents for Software Development. Also check out the demo video below:



“Eventually all our interactions with the digital world will be mediated by AI assistants. AI assistants will constitute a repository of all human knowledge and culture; they will constitute a shared infrastructure like the internet is today,” said Yann LeCun, one of the three godfathers of AI, in his talk at GenAI Winter School recently.

Brand Next talks about AI agent in its 2024 predictions. There’s a GPT / Copilot / AI agent for that - AI agents will become ubiquitous for specialized tasks, mirroring the app boom in the early 2010s. Remember when smartphones just came out about 15 years ago. Apple released there now iconic ad "There’s an App for that”. In 2024 we will see the same thing happen but with GPT’s and AI agents. Whatever specialized task you have from planning a holiday, to ordering groceries, to optimizing your web content, there will be GPT or AI agent specifically designed to help.

Venture capitalist Vinod Khosla, envisioned in his top 12 predictions that a future in which internet access will be mostly through agents. He predicted "There will be a billion+ programmers all programming in natural, human language, dramatically increasing what can be done with computers."

Demis Hassabis, the CEO of Google DeepMind, recently said in an interview with WIRED that he plans to combine large language models with the work his company has previously done training AI programs to play video games to develop more capable and reliable agents. “This definitely is a huge area. We’re investing heavily in that direction, and I imagine others are as well.” Hassabis said. “It will be a step change in capabilities of these types of systems—when they start becoming more agent-like.”

Meta CEO Mark Zuckerberg spoke about how if a business is trying to interact with a customer then the interaction is no longer limited to “the person sends you a message and you just reply”. It’s a multi-step interaction where the business would want to think through how it can accomplish the person’s goals. So, the job of the AI is no longer to just respond to the question.

“If someone else solves reasoning and we’re sitting here with a basic chatbot, then our product is lame,” he said, envisioning a kind of Meta AI general assistant product that will shift from something that feels more like a chatbot to things where you’re giving it more complicated tasks and then it goes away and does them.

“I think a big part of what we’re going to do is interacting with other agents for other people whether it’s for businesses or creators. A big part of my theory on this is that there’s not going to be just one singular AI that you interact with because every business is going to want an AI that represents their interests,” he added.

He further took the example of 200 million creators on Meta platforms and how they want to engage with their community but are limited by the hours in the day. He explains that if you could create something where that creator can basically own the AI, train it in the way they want, and can engage their community, then that’s going to be super powerful.



Despite the excitement surrounding AI agents, there are concerns and challenges that need to be addressed. Kailash Nadh, CTO at Zerodha, believes that while the future of AI agents running the internet is likely, it may not be a pleasant one. He points out that there are already instances of agents being used for malicious purposes, such as ordering pizza without permission. Additionally, Andrew Ng acknowledges that current AI agents are not fully reliable and can be finicky. However, he remains optimistic that with continuous iteration and improvement, these issues can be mitigated.


One of the most prominent areas of impact lies in enhancing automation and efficiency. AI agents can take over the reins of repetitive tasks such as data entry, scheduling, and report generation, liberating human employees to dedicate their time and expertise to more strategic endeavors. Similarly, these agents can optimize processes, manage complex supply chains, and even operate autonomous robots, leading to significant increases in productivity and reductions in operational costs.

Beyond automation, AI agents hold the key to crafting personalized and intelligent experiences. Customer service, for instance, can be revolutionized through AI-powered chatbots and virtual assistants, offering round-the-clock support, answering customer inquiries, and resolving issues with efficiency and accuracy, ultimately leading to enhanced customer satisfaction and loyalty.

Moreover, these agents can analyze user data and behavior to curate personalized recommendations for products, services, or content, tailoring experiences to individual preferences and creating a more engaging and satisfying user journey.

The education sector can also benefit immensely, with AI tutors adapting to individual learning styles and providing customized feedback, making education more effective and accessible to a wider range of learners.


AI agents are also catalysts for innovation and advancement. In the realm of research and development, they can analyze massive datasets, identify complex patterns, and generate hypotheses, accelerating scientific discovery in fields ranging from medicine and materials science to climate modeling. Additionally, autonomous robots controlled by AI can venture into hazardous environments and gather crucial data, expanding our understanding of the universe and paving the way for groundbreaking discoveries.

The potential of AI agents extends to addressing societal and environmental challenges. In healthcare, they can assist with diagnosis, monitor patient health, and even perform intricate surgeries, improving healthcare quality and accessibility for all. Environmental monitoring and protection can also be enhanced, with AI agents collecting data on pollution levels, tracking wildlife populations, and contributing to the sustainable management of natural resources. Furthermore, in disaster response scenarios, AI-powered robots can navigate dangerous environments, locate survivors, and provide crucial aid in the aftermath of natural disasters.

Emerging applications of AI agents continue to push the boundaries of possibility. Smart homes and cities can leverage these agents to control appliances, optimize energy consumption, and manage traffic flow, creating more efficient and sustainable living environments. The development of autonomous vehicles relies heavily on AI agents for navigation, obstacle avoidance, and real-time decision-making, paving the way for a future of safer and more efficient transportation. Even the creative industries are exploring the potential of AI for generating art, music, and written content, opening up new avenues for artistic expression and innovation.

The rise of AI agents is transforming the way businesses interact with customers and how people access the internet. From agentic workflows that deliver superior results to the development of platforms and real-world applications, the potential of AI agents is vast. While there are concerns and challenges to be addressed, the future of AI appears to be increasingly agentic. As frameworks and security measures evolve, the possibilities for AI agents to revolutionize various industries and enhance our digital lives are truly exciting. It is crucial for businesses and individuals alike to stay informed and prepared for this transformative shift in AI technology.