Stop Guessing, Start Seeing: Multi-Model Observability with LLMxRay 🕵️‍♂️
Have you ever wondered why the same prompt costs more in one language than another? Or why a model feels "smarter" in English but struggles with Arabic or Chinese? When working with LLMs, we often ...

Source: DEV Community
Have you ever wondered why the same prompt costs more in one language than another? Or why a model feels "smarter" in English but struggles with Arabic or Chinese? When working with LLMs, we often treat the response as a black box. We see the output, but we don't see the mechanics—the tokenization, the side-by-side comparison of different model families, or how different writing systems affect performance. I built LLMxRay to pull back the curtain. What is LLMxRay? LLMxRay is an open-source observability tool designed to help developers inspect how different LLMs handle the exact same prompt in real-time. Whether you are using local models via Ollama/LM Studio or cloud-based APIs, LLMxRay gives you a "side-by-side" X-ray view of your prompt's journey. Why use it? Multi-Model Comparison: Run one prompt against multiple models simultaneously. See how Llama 3 compares to Mistral or GPT-4o in one view. Multilingual Deep-Dive: This was a big focus for me. The tool supports 4 languages: Engli