Navigating AI Integration from OpenAI to Qwen: A Personal Reflection
Reflecting on my journey with Arise & Shine Transporters, I learned valuable lessons about the integration of AI tools and the importance of cost modeling before going live.
What was hard in building Arise & Shine Transporters? Starting from scratch for a project like this is challenging. Coming into it without knowledge of how Virtual Private Servers (VPS) worked, I found myself needing to learn SSH commands, Linux administration basics — all while trying to set up the environment needed for my application. The initial setup phase was more about understanding and working with command-line tools than just clicking buttons on a UI interface.
Early architecture flaws: Early in our journey, there were many mistakes made along the way due to an immature codebase and poor architectural choices. One of these challenges was wiring AI functionality like article writing directly into OpenAI's API during development. This turned out to be financially unsustainable as we progressed to production. Running this in real-time with a high volume meant that costs rapidly escalated, making it clear that cost modeling needed to happen before going live.
Migrating from Contabo to Hetzner: After realizing the limitations of our hosting provider (Contabo), I had to switch providers. This involved moving workloads and ensuring stability across different VPS environments. Switching from Contabo’s managed PaaS model, which is easy but not flexible for complex projects like mine, led me to choose Hetzner's Virtual Private Servers (VPS) based on its specs and pricing that better fit our workload needs.
Switching OpenAI API to Qwen: The final nail in the coffin was switching from the original OpenAI API to a self-hosted model. Initially thinking about integrating an AI solution like this, I started with OpenAI's API because it seemed like the go-to option for generative tasks such as article writing and report summarisation. However, running these directly on OpenAI’s servers proved financially prohibitive once we scaled up. This led me to explore alternatives: self-hosting a model (Qwen), which is significantly cheaper at steady state compared to using an API with OpenAI.
Skills gained: These real-world experiences taught me several important lessons: - SSH and remote server administration from zero, including VPS provisioning. - Linux VPS management such as environment setup and process supervision through PM2. - Real integration of third-party APIs like Protrack 365 telemetry, Nominatim geocoding services, OpenAI-compatible interfaces for AI models. - The value in having provider-agnostic architectural decisions — the abstraction layer over API integrations made switching to Qwen seamless without touching business logic.
Lessons worth carrying forward: These experiences highlight how crucial it is to consider real-world constraints when building applications. It also underscores why picking the right hosting and AI providers, like migrating from Contabo to Hetzner for VPS resources or choosing a self-hosted model over an expensive API, can make all the difference in project success.
Future outlook: The system now runs smoothly with live production data handling real users' needs. While there are still rough edges visible (like needing SSH commands and Linux administration), it’s reassuring to know that these technical challenges were overcome along the way.
