Posts

The Celery Question For years, when someone asked “How do I handle background tasks in Django?”, the answer was almost always “Use Celery”. And for good reason. Celery is mature, battle-tested, and has solved async task processing for countless Django applications. It has extensive documentation, a large ecosystem of plugins, and most Django developers have at least some experience with it. But Celery also brings complexity. You need a message broker like Redis or RabbitMQ. You need to manage worker processes. You need to implement your own progress tracking if tasks need to report status. You need to handle state persistence separately if your tasks need to remember things across restarts. As your application grows, this infrastructure overhead grows with it. ...

The Challenge of Reliable Background Operations Building web applications with Flask is straightforward until you need to handle operations that can fail. Database operations timeout, external APIs become unavailable, and network connections drop. These failures are inevitable in production systems, yet handling them gracefully often requires significant engineering effort. Recently, I was trying to build a Todo application where users could create, update, and delete tasks. On the surface, this seems simple. But when you start thinking about production scenarios, questions emerge. What happens if the database connection fails mid-operation? How do you handle retries without duplicating operations? What if the server crashes while processing a delete request? ...

The Problem: Connecting AI Assistants to Your APIs As AI assistants like Claude and VS Code Copilot become more powerful, developers are discovering a critical bottleneck: these AI tools can’t directly interact with our existing REST APIs. After building Django DRF to MCP converters and Kafka MCP servers, I realized a fundamental challenge: Every API requires custom MCP server implementation. You shouldn’t need to write custom code for every API integration. You shouldn’t have to learn Model Context Protocol’s internals just to connect Claude to your weather API, GitHub API, or internal company APIs. ...

The Problem: Getting Real-time Data into Kafka Have you ever tried to stream Wikipedia changes or other real-time Server-Sent Events (SSE) updates into Kafka? If you have, you probably wrote custom Kafka producer and consumer code to handle these events, managed reconnections, and built your own error handling. There had to be a better way. Introducing the SSE Kafka Connector I’m excited to share my new open-source project: a generic Kafka Connect connector for Server-Sent Events. It lets you stream data from any SSE endpoint straight into Kafka topics—no custom code required. ...

The Problem: When APIs Meet AI Assistants A few weeks ago, I was experimenting with Model Context Protocol (MCP) for a Kafka integration project. The experience was eye-opening - being able to interact with complex systems through natural language felt like a glimpse into the future of development workflows. After successfully getting MCP working with Kafka, I started thinking about other systems I could connect. That’s when I looked at one of my old Django projects. ...

What if I told you that you could build, host, and maintain data APIs completely free using just GitHub? Here’s how I built a production-ready data platform without spending a penny on hosting. The Problem: APIs Are Expensive (Or Are They?) Building data-driven applications usually means expensive cloud hosting, database costs, and server maintenance. Most developers assume they need AWS, Google Cloud, or Azure to run automated data collection and serve APIs. But what if there was a better way? ...

Ever wondered where those massive icebergs the size of cities are drifting? I built a system to track them in real-time using NASA satellite data. The Problem: Lost Giants in the Southern Ocean Antarctic icebergs are some of the most fascinating phenomena on Earth. These floating ice mountains—some larger than entire countries—break off from ice shelves and drift across the Southern Ocean for years. But tracking their movements has always been challenging. ...

Integration testing is crucial for Kafka Connectors to ensure they work correctly with external systems. This guide provides a comprehensive approach to setting up and executing integration tests for Kafka Connectors, covering everything from environment setup to best practices for testing source and sink connectors.

A step‑by‑step guide to using FastMCP and the MCP protocol to expose Kafka operations (topic management, produce/consume, troubleshooting) as LLM‑accessible tools.

Tired of manually reformatting Markdown for Medium? Discover my open-source tool that converts Hugo-friendly Markdown into perfectly formatted Medium posts instantly.