Request help for Building a chatbot for Log analysis using LLMs The Motivation is as follows: There are many industrial applications that use various systems (devices). It is these logs that are analysed by the engineers to study the functioning, performance and maintenance. This exercise is resource intensive. But it is unavoidable because there is deep reasoning and logic behind the analysis and is based on the industry specific knowledge and domain expertise and is currently done by humans.For standard devices like our computers, we have system logs, event logs and application logs. For analysing these logs, there are many commercial and open source applications since these logs follow a standard format. However, for proprietary systems like the one I am building for, the logs follow a custom format and these formats may have custom entities and descriptions. Of course, there is a document that has the Business logic and knowledge embedded in the logs that explains these custom logs in detail i.e. events codes, Alarms, descriptions, etc. This coupled with the "How to perform log analysis" document and product design document will help to understand why a device behaved in a certain way. My main goal is to provide a chatbot application that can use an LLM to extract key insights from log files. Essentially, provide a simple user interface where the upper management (Non technical staff) can ask questions and get a response. i.e. 1. When was the last time Device A generated the following events/alarms ? 2. In the last 30 days, how many times did Device B log the following error "<error description> 3. When the <error> was logged by Device A, was Device B in working condition? 4. Why did Device C generate the alarm at <time> on <date>? 5. Generate an error report for Device D in a tabular form with date time and the event with causation and its consequences on other devices? 6. Generate a summary report for the entire system for the month of <month>?

Imagine building an App that simulates arguments in a court case between a prosecutor and the defense counsel with the judge listening and who finally makes a judgement on the case. To keep it truly unbiased, we use independent Agentic frameworks and LLMs. The prosecution is built with agents using AutoGen and Claude, the defense uses CrewAI and Gemini Pro and the Judge Agent is built using GPT-4o and LangGraph. The knowledge base is a common case file that is available to both sides. Based on the arguments, the agentic Judge decides on the final verdict. To push it a bit further, we could have 10 jury members (agents) who will also listen to the case offer their unanimous decision. So, Is this possible in the current state of AI ?

A local university wants me to develop a RAG enabled chatbot to query a large number of thesis/dissertations in their library. The documents are in PDF format with images, tables and charts. The total is around 3500 documents and more will be added as students graduate. request any suggestions to handle this heavy load. Any pointers to articles/videos online would be most helpful Thanks.

Looking for a open/closed source solution for LLMs doing the work of Data Analysts on really large databases. For example, The Microsoft Contoso Database used as a demo for learning Power BI is approximately 450 MB and contains retail sales data. It has tens of tables each with several tens of columns. Is there a solution presently for querying in natural language and getting the LLM to perform data analysis. Essentially replacing a data Analyst😉