Analyze Images, Videos, Documents & Audio with Gemini Tools and Qwen LLM Agent

📁 Analyze uploaded images, videos, audio, and documents with specialized tools — powered by a lightweight language-only agent.

---

🧭 What It Does

This workflow enables multimodal file analysis using Google Gemini tools connected to a text-only LLM agent . Users can upload images, videos, audio files, or documents via a chat interface. The workflow will:

• Upload each file to Google Gemini and obtain an accessible URL.
• Dynamically generate contextual prompts based on the file(s) and user message.
• Allow the agent to invoke Gemini tools for specific media types as needed.
• Return a concise, helpful response based on the analysis.

---

🚀 Use Cases

• Customer support : Let users upload screenshots, documents, or recordings and get helpful insights or summaries.
• Multimedia QA : Review visual, audio, or video content for correctness or compliance.
• Educational agents : Interpret content from PDFs, diagrams, or audio recordings on the fly.
• Low-cost multimodal assistants : Achieve multimodal functionality without relying on large vision-language models .

---

🎯 Why This Architecture Matters

Unlike end-to-end multimodal LLMs (like Gemini 1.5 or GPT-4o), this template:

• Uses a text-only LLM (Qwen 32B via Groq) for reasoning.
• Delegates media analysis to specialized Gemini tools .

✅ Advantages

Feature	Benefit
🧩 Modular	LLM + Tools are decoupled; can update them independently
💸 Cost-Efficient	No need to pay for full multimodal models; only use tools when needed
🔧 Tool-based Reasoning	Agent invokes tools on demand, just like OpenAI's Toolformer setup
⚡ Fast	Groq LLMs offer ultra-fast responses with low latency
📚 Memory	Includes context buffer for multi-turn chats (15 messages)

---

🧪 How It Works

🔹 Input via Chat

• Users submit a message and (optionally) files via the chatTrigger .

🔹 File Handling

• If no files: prompt is passed directly to the agent.

• If files are included:

  • Files are split, uploaded to Gemini (to get public URLs).
  • Metadata (name, type, URL) is collected and embedded into the prompt.

🔹 Prompt Construction

• A new chatInput is dynamically generated:

   User message
   Media: [array of file data]
  

🔹 Agent Reasoning

• The Langchain Agent receives:

  • The enriched prompt

  • File URLs

  • Memory context (15 turns)

  • Access to 4 Gemini tools:

    • IMG : analyze image
    • VIDEO : analyze video
    • AUDIO : analyze audio
    • DOCUMENT : analyze document

The agent autonomously decides whether and how to use tools, then responds with concise output.

---

🧱 Nodes & Services

Category	Node / Tool	Purpose
Chat Input	chatTrigger	User interface with file support
File Processing	splitOut , splitInBatches	Process each uploaded file
Upload	googleGemini	Uploads each file to Gemini, gets URL
Metadata	set , aggregate	Builds structured file info
AI Agent	Langchain Agent	Receives context + file data
Tools	googleGeminiTool	Analyze media with Gemini
LLM	lmChatGroq (Qwen 32B)	Text reasoning, high-speed
Memory	memoryBufferWindow	Maintains session context

---

⚙️ Setup Instructions

1. 🔑 Required Credentials

• Groq API key (for Qwen 32B model)
• Google Gemini API key (Palm / Gemini 1.5 tools)

2. 🧩 Nodes That Need Setup

• Replace existing credentials on:

  • Upload a file
  • Each GeminiTool (IMG, VIDEO, AUDIO, DOCUMENT)
  • lmChatGroq

3. ⚠️ File Size & Format Considerations

• Some Gemini tools have file size or format restrictions.
• You may add validation nodes before uploading if needed.

---

🛠️ Optional Improvements

• Add logging and error handling (eg, for upload failures).
• Add MIME-type filtering to choose the right tool explicitly.
• Extend to include OCR or transcription services pre-analysis.
• Integrate with Slack, Telegram, or WhatsApp for chat delivery.

---

🧪 Example Use Case

"Hola, ¿qué dice este PDF?"

Uploads a document → Agent routes it to Gemini DOCUMENT tool → Receives extracted content → LLM summarizes it in Spanish.

---

🧰 Tags

 multimodal, agent, langchain, groq, gemini, image analysis, audio analysis, document parsing, video analysis, file uploader, chat assistant, LLM tools, memory, AI tools

---

📂 Files

• This template is ready to use as-is in n8n.
• No external webhooks or integrations required.