Multimodal VQA Agent

Visual Question Answering

GenAI agent that answers natural language questions about images using multimodal architecture.

PythonBLIP-2OpenAIHugging FaceStreamlit
View on GitHub

The Problem

Users need to ask questions about images (accessibility, education, customer service) but most AI systems only handle text. Bridging vision and language requires specialized architectures.

The Solution

A multimodal agent combining BLIP-2 for image understanding with LLMs for language reasoning. Includes RAG components for contextual enrichment and prompt chains for multi-step reasoning.

  • ▶BLIP-2 + LLM multimodal architecture for image understanding
  • ▶RAG components and prompt chains for contextual reasoning
  • ▶Real-time Streamlit interface for live testing and demos
  • ▶Evaluated on general and domain-specific image-question datasets

System Architecture

Loading diagram...

Results

✓Handles natural language questions about arbitrary images
✓Applicable to accessibility, education, and image-based customer service
← All ProjectsWork with me