Microsoft Agent? · AI Automation Society

Microsoft Agent?

Hey Nate, hey awesome other guys,

I love what you do! 2 months or so I switched over from make, where I coded self-improving, self-extending, user-centric multi-step-bots, but this took like forever and hundreds of modules. With n8n this is just so much easier.

What I try to create since a while is a similar assistant to your 2.0, but with Microsoft instead of Gmail. It's getting better after I found out that credentials were taken wrongly first, but now I face other issues.

What is core here? Even within email, there are so many possible tasks I want the assistant all to be able to combine effectively and efficiently. Commands like "can you please summarize the communication with xyz", "are there important mails in my mailbox", "can you please draft a response to xyz including abc", etc. should be doable. Initially with Telegram, later I'd love to install yor 11Labs integration <3

The above I programmed with make like: 1) understand the final goal of my task, 2) adapt it based on the knowledge about me (vector or knowledge database), 3) search API for final step in self-created API-overview, 4) feedback if there is no final API / create a "project" if there is one, 5) iteratively approach the goal with the bvest fitting API in the list to get closer to the goal, 6) if there is no API helping further: 6a try to reserch API-calls, try one out and integrate it in the API list if it works, or 6b ask the user if there is none. 7) asynchron at nights: review missing skills based on the user demands and try to find some with "more complex methods" (such as combining purposely created hallucinations with vector based hallucination on API-vactor base only, response/feedback of user, trying ou and proactively using disclosures. You can imagine what a code monster this was at n8n, considering I built this all for several systems, incl. lots of MS integration such as Outlook, Tasks, Onedrive, but also other services I use.

With n8n I find myself having a harder time. Of course, the "fromAI" function is so massively helpful, bevor this was a nightmare (since I tried to do all this without any additional AI usage to have faster responses), also the AI agent modules make it a lot easier. But when I e.g. try to adapt your 2.0 assistant and need the agent to structure its approach more, I get errors.

Example: "archive my mail from peter regarding ai" needs the ai to do those steps: 1) get all mails matching those criteria (for the message ID needed to move it), 2) move mail. But sometimes the system doesn't "want to" take the ID first and makes up something or uses some other id. It seems, the prompting either underfits or overfits the agent's input.

I think, ideally the agent is built like an organization:

1) customer contact (understand him, process what he needs, constantly updating him during the process, etc.)

2) "project manager" creating a project plan with tasks, distributing them, adapting the plan based on task results, making decisions (like pulling in the user if needed) based on availible resources and an overall goal (like "serve the user, reduce his workload, improve his life quality, etc.)

3) "departments" (such as email, calendar, etc.) for specific tasks, reporting back to the PM (if needed with sub departments for special tasks, but I have no idea yet if this is needed)

4) "research & governance ", where departments can place things they want to learn (for the research agent to asynchron try to fill those gaps in APIs, but also analyze prompts etc. of other department to make it more efficient overnight)

5) customer contact informing the user about the result, needed input, further suggestions, etc.

optionally:

0) coach (trying to create goals for the user to support mental wellbeing, relaxation, maybe even use smartwatch data etc. for this, derive long-term goals, "talk" to the PM to fit those goals in if feasible timely etc.)

What do you think about such a thing in n8n? I currently miss the right mix of overfitting functions (having functions at the agent for every task, what improves correct usage, but limits functions) and underfitting them (having like 3 functions that get adjusted for every usage) and didn't find a good way, yet. Maybe because I yet use o4, which isn't capable enough. I think V3 could be nice, but considering where the data go, I rather not connect this with all my cruicial data ^^

What are your thoughts' Can't wait to get your ideas and maybe a template @Nate ^^, since I try for a month now and got stuck.

Thank you so much guys! :)

12 comments