Aside from PydanticAI, Agency Swarm could be a possible integration for orchestration of AI agents. I have been playing with this of late and have had good results.
I have created an AI agent in to scaffold Quasar Framework projects (agent has a cli interaction tool) based on project requirements passed down from a frontend architect agent. If there is interest in this I can provide more details as to what I am doing in agency swarm that may shed some light as to my thought process and goals using this framework.
Yeah I have heard good things about Agency Swarm too! That would be so cool if you want to share more what you are doing with it!
Hi Cole,
Sure. I’ll go into more detail.
I’m creating an AI ran development agency using AI agents to develop applications (web, mobile, and OS).
So far my agency consists of a CEO, project manager, frontend architect, backend architect, copywriter, designer, Quasar developer and React developer (I plan to add other framework developers with time). Each developer has their expertise, as human development agencies do, that focus on the frameworks and libraries in their vertical.
The current limitations to agency swarm is that it currently only communicates with openAI or a custom model. I plan on forking the repo adding the capability to use both.
It also relies on Assistants API v2, so I planned on forking the Astra Assistants API, which has been open sourced, to allow for other vector db frameworks and not just AstraDB cloud service.
How I see this integrating with Bolt.diy is in two different ways:
- using n8n for workflow control that communicates with the agency
- directly integrating Bolt.diy into the agency as the UI for the agency
To elaborate further on each method…
Using n8n with the agency would not only allow for enhanced agency processes, but would allow the agency to fully implement RAG connected to an agency vector store for updates to knowledge on frameworks, agency default project templates and fill in gaps with design methodologies. It also creates, or could create, the ability to enhance agentic behavior and advanced reasoning by means of introducing workflows that the agents can interact with themselves to reduce hallucinations by proofing tasks when they are completed. It could provide agents access to online closed models for image generation and other abilities the agency could require if not hosted locally. I have an idea how this would work, though I haven’t gotten this far yet.
Directly integrating Bolt.diy into the agency could allow the user to interact with the agency similar to above, though could either require more user interaction with the agency or require more coding for agent tools.
As of right now, I’d say the preferred method would be n8n.
Love this @MrMarino, congrats on having a successful agency!
Don’t like those limitations of Agency Swarm haha, but that’s cool you are forking it to make it fit to your needs.
I like where your head is at with n8n and integrating directly with bolt.diy too! Anything you can do in n8n you could do with Python code to go along with Swarm/another agent framework, but am I correct in assuming you are talking about n8n just because it makes things a lot easier to set up?
@ColeMedin
Good question. The n8n framework will have a few purposes.
The first is providing easy web hooks to the agency to communicate with the agents. Agency swarm uses an @{agent name} to communicate directly to agents in the swarm / agency. Leaving out the @{agent name} will send the commination to the CEO of the agency.
Furthermore I find it may be beneficial to have an agent in the middle, so to speak, between the user and the agency to facilitate prompt priming and modifications to a users prompt before sending it further on to the agency. This agent would be responsible for verifying the users true intent through it’s communication with the user and then formulating the prompt to be sent to the agency. N8N makes this extremely flexible and dynamic. One could argue that the CEO agent could be coded to do this, but in my opinion the CEO agent’s responsibility is knowing and running the agency. By separating this, I believe this allows the agency to focus on it’s purpose with clear prompts hopefully reducing any agent confusion. Both the n8n instance and agency will have access to the same vector store / db separately so that the agency isn’t reliant on n8n for information.
Then there is rapid prototyping of tools for agents, which will eventually be brought over to the swarm framework as an agent tool in python. For me, this means that once the tool is prototyped, up and running, there will be time to migrate it to the agent without there being any downtime (at least I hope this is the case. I can’t see a reason as of now why this wouldn’t be possible).
How I see this happening, or at least my idea, is that once the prototype is created and vetted, the tool would be migrated to the agent that would use it. Testing could begin with the above hook and directly communicating with the agent to see how it performs while the prototype is used until cutover. Caveats regarding prompts can be handled and if a specific communication style is needed for the agent to utilize the tool effectively is determined, this could then be added to the vector store / db to which the CEO would then pickup to further influence the agent in the right direction. The agent would then access the the same prompt information from the store when receiving the directive to validate directive adheres to the tools usage. My hope is that this creates a sort of check and balances for agents and tools while still allowing the agent the freedom of reasoning.
As you can probably tell, I’m a fledgling when it comes to AI and agents. The above, for now, is all hypothetical until I have a better grasp on AI capabilities and start to implement it. It may be the wrong type of thinking, but I see n8n as not just an easy way of setting things up, but a potential for it to be a dynamic communication platform enabling concise user and agency / agent interactions. I did briefly look at flowise, but my impression is that it is more constrictive, adhering to a defined set workflow process flow. Please pardon my ignorance with all this.
@ColeMedin I’ve created a repro of the modified astra assistants api that has supabase support: GitHub - Karnith/assistants-api
I’m starting testing of it, but so far it seems stable. I’ve made a lot of enhancements, especially when it comes to tools. I’ve created a unified tooling system that will ensure all llm models will be able to use tools.
It’s a work in progress and I’ve created detailed documentation of the system and client to it.
Wow this looks great, fantastic work!!