ProjectsSeptember 15, 2026

VirtuJudge

Ahmed Elsherbiny
Omar Salama
Mohamed Abbas
Mohamed Adel
Ayman Yasser
In competitive environments such as hackathons, startup pitch competitions, and academic presentations, competitors rarely receive objective, structured feedback before facing actual judges. Traditional practice methods rely on ad-hoc human reviews, which are inconsistent, difficult to schedule, and lack quantitative metrics on vocal delivery, slide coverage, and team participation balance. VirtuJudge bridges this gap by providing an autonomous, competition-grade virtual judging system. The platform analyzes presentation video recordings and slide decks in parallel, tracks active speakers using multimodal audio-visual signals, generates grounded challenge questions using Retrieval-Augmented Generation (RAG), and delivers comprehensive evaluation reports where every score and critique is directly linked to timestamped video evidence and specific slide pages.
  • Multi-Tenant Workspace & Team Collaboration:
    Role-based access control (Owner vs. Member) allowing team collaboration on pitch projects, session management, secure member invitations, and ownership transfers.
  • Direct-to-Storage Resilient Ingestion:
    Offloads heavy video and slide deck uploads from the backend by issuing time-limited presigned URLs for direct transfer to S3 / Cloudflare R2 object storage. Client-side Web Workers compute streaming SHA-256 chunk checksums before upload to ensure end-to-end payload integrity.
  • Real-Time Telemetry & Progress Streaming:
    Live pipeline execution updates delivered over Server-Sent Events (SSE), featuring monotonic event sequencing and automatic stream recovery via Last-Event-ID to withstand transient network drops.
  • Multimodal Speaker Diarization & Active Speaker Attribution:
    Combines acoustic diarization with computer vision lip-motion tracking (Lip-MAR) to isolate individual presenters (SPEAKER_00, SPEAKER_01) and provides an interactive interface for team members to map themselves to their respective presentation segments.
  • Context-Grounded Follow-Up Q&A:
    Synthesizes pitch transcripts with presentation decks and supporting documents to generate challenging, context-grounded questions. An interactive browser-based voice recorder captures team answers with automated transcription and quality analysis.
  • Evidence-Linked Rubric Scoring & Exportable Reports:
    Scores pitch sessions across standard competition rubrics (delivery, problem-solution fit, technical depth, and Q&A responsiveness), anchoring every critique to specific video timestamps and slide numbers. Generates downloadable PDF reports for offline team review.
  • Team Owner:
    Full administrative control over the team workspace. Can create and delete projects, manage team members, transfer ownership, initiate pitch evaluations, and execute privacy-preserving data erasure requests.
  • Team Member:
    Can view workspace projects, upload practice recordings and decks, review telemetry, participate in interactive voice Q&A sessions, and access final evaluation reports.
  • Workspace & Asset Preparation:
    Team members select or create a project workspace and select the target pitch recording (MP4/WebM) along with presentation slides (PDF).
  • Direct Signed Upload:
    The browser calculates a SHA-256 chunk checksum in a background Web Worker and requests presigned PUT URLs from the backend, uploading media directly to object storage without placing transit load on the API server.
  • Multimodal Analysis:
    The backend initiates an asynchronous AI pipeline that extracts audio tracks, executes speech-to-text transcription, runs audio diarization, tracks presenter lip movement via computer vision, and parses slide content.
  • Interactive Speaker Mapping:
    The team reviews identified speaker clusters alongside audio and visual previews, attributing each detected presenter segment to verified team members.
  • Grounded Follow-Up Q&A:
    Using vector embeddings and Retrieval-Augmented Generation (RAG), the system generates grounded judge questions. Presenters record their answers directly in the browser.
  • Evidence-Linked Scoring & Report Generation:
    The evaluation engine scores both the pitch delivery and Q&A responses against structured competition rubrics, compiling an evidence-backed report with timestamped video moments and slide references.
  • Frontend
    • Next.js (App Router): Modern React server components, dynamic routing, and API rewrites.
    • React & TypeScript: Interactive, accessible UI built with strict static typing.
    • TailwindCSS: Clean, responsive glassmorphic interface design system.
    • TanStack React Query: Efficient server-state synchronization, caching, and optimistic mutations.
    • Server-Sent Events (SSE): Low-latency unidirectional event streaming for real-time inference tracking.
    • Web Workers: Multi-threaded client-side SHA-256 chunk hashing for pre-upload validation.
  • Backend
    • FastAPI (Python): High-throughput asynchronous REST API control plane.
    • Clean Hexagonal Architecture: Strict four-layer design (Domain, Application, API, Infrastructure) separating domain invariants from framework dependencies.
    • SQLAlchemy (Async): Fully asynchronous database operations and Unit-of-Work patterns.
    • Pydantic: Fast data validation, schema enforcement, and OpenAPI generation.
  • AI/ML & Media Processing
    • Speech-to-Text & Diarization: Audio extraction, transcription, and speaker clustering.
    • Computer Vision (Lip-MAR): Lip-motion active speaker detection for video-audio alignment.
    • Vector RAG: Document parsing, semantic chunking, and grounded question generation.
    • FFmpeg & FFprobe: Media stream verification, audio waveform extraction, and format sanitization.
  • Database & Storage
    • PostgreSQL & pgvector: Relational data persistence, transactional integrity, and vector similarity search.
    • Redis: Asynchronous AI job queue and low-latency event cache.
    • Cloudflare R2 / AWS S3 / MinIO: Object storage for pitch media, slides, answer recordings, and generated reports.
  • Direct-to-Cloud Ingestion with Integrity Guarantees:
    Eliminated backend bandwidth bottlenecks by routing large video uploads directly to S3/R2 presigned URLs. Solved payload tampering risks by computing SHA-256 digests in browser Web Workers and enforcing strict storage-level checksum verification upon completion.
  • Resilient Real-Time Event Streaming:
    Addressed connection interruptions during multi-minute AI inference pipelines by designing an SSE streaming protocol with monotonic event sequence IDs. On reconnect, the client provides Last-Event-ID, allowing the backend to replay missed events from Redis cache without loss of state.
  • Multimodal Presenter Attribution:
    Bridged acoustic diarization with computer vision lip-motion analysis to disambiguate overlapping speakers and assign presentation intervals accurately to individual team members.
  • Optimistic Concurrency Control (OCC):
    Safeguarded concurrent multi-user team workflows by implementing HTTP ETag and If-Match preconditions on mutable resources, preventing lost updates when multiple team members collaborate simultaneously.
  • Domain-Driven Architectural Isolation:
    Enforced strict boundaries between the domain layer, use-case application services, HTTP routes, and infrastructure adapters, ensuring the core business logic remains independent of external frameworks and third-party services.
VirtuJudge provides teams with an end-to-end, high-performance virtual judging platform that transforms competition rehearsal into an objective, data-driven process. By combining direct cloud storage ingestion, real-time telemetry streaming, multimodal speaker attribution, and evidence-grounded evaluation rubrics, the platform delivers actionable feedback that elevates competitor readiness for high-stakes presentations.

Related projects

IEEE ZSB Website

IEEE ZSB Website

Ahmed Elsherbiny
Mohamed Emad
Omar Salama
Development of the official IEEE ZSB (Zagazig Student Branch) website — a modern, responsive platform built to represent one of Egypt's most active IEEE student branches.
Full-Stack ROV Control System

Full-Stack ROV Control System

Mohamed Abbas
Ahmed Elsherbiny
Development of a dual-mode control system for a competitive Remotely Operated Vehicle, featuring a React/TypeScript frontend and a Node.js backend with ROS integration.