At Deepgram we want to meet developers where they are in their journey to building Voice AI products and features by offering a range of open source projects to help bootstrap your development efforts, including 4 SDKs (Software Development Kits) in the following languages:
Lately in our Developer Communities (Github Discussions + Discord), we’ve noticed a lot of questions about lower-level WebSockets as they relate to the Streaming API. One of the many SDKs' objectives is to make them easier to use by removing the burden and requirement to understand WebSocket complexities. It’s in those objectives that we feel our SDKs could provide immense benefit versus the alternative, a developer trying to build one from the ground up. We encourage Deepgram developers to embrace open source in the spirit of DRY (don’t repeat yourself).
This blog post is less about SDKs and more about the philosophies of open source software (OSS) and the discussions of code reuse and community contributions. The hope is to steer individuals clear of reinventing the wheel because there are numerous options available to achieve your objectives thereby finishing your app, but reimplementing an SDK should be among the last in your toolbelt as a developer. One of the first tools should be to embody the spirit of open source.
The hope is to discuss the many challenges faced with duplicating an existing project in this blog post and why it might be better to embrace open source, code reuse, and contribute to a project that already exists. Before we start, let's level set on the terminology.
Terminology
Upstream Project: The original source code repository from which derivatives, or "forks," are created. It serves as the primary and authoritative version of the project.
Forked Project (or Fork): A copy of the upstream project that exists independently, allowing developers to make changes or enhancements without affecting the original project. Forks may diverge significantly over time.
Parallel Project: A separate initiative that runs alongside another project, typically aiming to achieve similar objectives but is developed and managed independently.
Challenge: Project Planning and Being Left Behind
One of the more significant issues with creating a project based on a Platform owned by another organization is you aren't plugged into what releases and features will be coming (i.e., the roadmap). The upstream project benefits from an orchestrated development plan, encompassing everything from minor feature tweaks to major releases. This roadmap isn't just a schedule; it's a strategic vision that guides the project's evolution, informed by the community's immediate needs and the technology's long-term goals.
For contributors and users of the project, this roadmap offers a glimpse into the future, allowing for better planning, anticipation of new features, and alignment of individual projects with upcoming enhancements. Moreover, being part of the upstream project can sometimes grant early access to alpha or beta releases, providing a valuable opportunity to influence the final product through feedback and contributions. This insider advantage is not just about having a sneak peek; it's about actively shaping the technology you rely on.
On the flip side, attempting to replicate or fork an existing project without this insider perspective places one in a perpetual state of catch-up. Without access to the project's strategic roadmap and early releases, Parallel Projects find themselves reacting to developments rather than anticipating them. This reactive posture not only hinders the ability to innovate but also multiplies the effort required to maintain parity with the upstream project. Look up Sisyphean task for more details.
Challenge: Implementation Resources
One of the most compelling advantages of contributing to an upstream open-source project is the robust support and stability provided by community members. Some of these members are compensated or are employed by organizations to dedicate their time to the project. This arrangement is a testament to the project's strategic importance to the organization, ensuring a steady influx of resources and attention that fosters both growth and reliability.
These community members, often experts in their fields, not only contribute code but also offer guidance, review contributions from the broader community, and help maintain the project's direction and quality. Their involvement guarantees a level of support and development activity that lone projects or smaller, volunteer-driven initiatives struggle to match. This continuous investment in the project's health means that when you build on top of such an ecosystem, you're working with individuals who want you to be successful, benefiting from a stable and cutting-edge foundation.
When contributors know that their efforts are supported by organizations committed to the project's success, it creates an environment of trust and collaboration that attracts more contributors. This cycle of contribution and improvement makes the project more resilient and adaptive to changes in technology or user needs.
Challenge: Getting Help and Support
The strength of an open source project often lies in its community. A vibrant and healthy community means guaranteed support for new and seasoned developers. This support goes beyond mere code contributions; it encompasses bug tracking, feature requests, documentation, and real-time assistance for those tricky issues that can stall development.
When you contribute to or rely on an established upstream project, you tap into a collective intelligence and resource pool that individual projects, especially those managed by a single person or a small team, simply cannot match. This network acts as a safety net, ensuring that questions are answered, improvements are continuously integrated, and knowledge is freely shared. The assurance of community support mitigates the risk of project abandonment, ensuring that the project remains resilient over time, adapts to new challenges, and evolves with the technological landscape.
Conversely, projects that hinge on the efforts of a Lone Wolf face a precarious dilemma. While the initial enthusiasm and dedication of a single individual can be admirable, it's inherently fraught with risks. Commitments can change, resources are finite, and the capacity for support is limited by time and energy.
This scenario creates a single point of failure, where the project's sustainability is vulnerable to life's unpredictability. If that individual steps away or their circumstances change, the project risks becoming stagnant or, worse, obsolete. Knowing this, would you bet your project's well-being or very existence on a project that could simply disappear without any notice?
Challenge: Poor Adoption
In the open-source ecosystem, a project's success and widespread adoption are not just about its functionality; they hinge on its ability to meet and exceed community expectations. Parallel projects, especially those spearheaded by individuals without the backing of a larger community, face an uphill battle in gaining traction. For these projects to justify their existence alongside or in place of an established upstream project, they must offer significant advantages or innovations that the original does not.
This goes beyond mere functional equivalence; it requires a unique value proposition that clearly addresses unmet needs or introduces novel approaches that significantly improve upon the status quo. Without this differentiation, potential users and contributors will likely stick with the familiar, well-supported upstream projects, which already enjoy the benefits of a broad user base, extensive documentation, and a track record of ongoing support and development.
In the context of contributing to the upstream project versus going it alone, the former often presents a more viable path to making a lasting impact. By channeling efforts into an established project, contributors can leverage the collective strength and visibility of the community to drive changes that benefit a wider audience. This collaborative approach not only maximizes the chances of adoption but also aligns with the open-source ethos of building together, where the sum of our collective efforts leads to greater innovation and utility than any one of us could achieve alone.
When To Go Lone Wolf
There are definite circumstances in which you should turn Lone Wolf and fork or begin a new Parallel Project. Let's take a look at those tipping points.
A Novel Idea - When you have an idea that is truly different from a given project or experience. Now, this idea isn't novel just because you wrote it. If the functionality does the same or similar thing, this isn't new or novel. Suppose you just swapped technologies, like a database or messaging platform. In that case, it will take a lot of work to convince someone to take the time, effort, and money lost in reimplementation to make the switch without significant material improvement in making that change.
Unaccepted Changes - If your contributions hold substantial value but you can't agree with the maintainers to get some semblance of the idea, feature, or contribution into the project. You need to ask yourself if it is worth taking on the responsibility, effort, and energy to support a Forked or Parallel Project.
Commercial Ventures: Unfortunately, we need food and shelter to survive, so money inevitability enters the equation. Under the proper open source license, a commercial offering based on OSS is an option. In this case, a fork is intentional because you provide some added value (aka material difference). The goal in this scenario is to make it so your Fork is easily updated from the Upstream Project.
Learning - Sometimes, the goal is to learn about a particular technology, like WebSockets. Usually, those pursuits are based on some need or problem you are trying to solve. The advice here is not to duplicate an existing project implementation (ie some existing WebSocket project) but rather implement something new and unique to your problem. Maybe with the intent of creating a new and unique open source project!
Where To Go From Here
We have all heard the phrase "imitation is the sincerest form of flattery," but when it comes to software, imitation is a thinly veiled trap distracting you from working on your actual problems or goals. It might indeed take a little upfront effort and time to understand someone else's code or Upstream Project, but the long-term savings using code that a community is going to maintain is how you are going to free up your time to work on more interesting problems.
A large part of this is possible through the power of open source software. It's not merely a development model; it's a testament to the power of the collective endeavor of people. By embracing this, we contribute to the advancement of a purpose and partake in a shared journey of continuous learning and mutual empowerment. Open source is the true embodiment of "we are greater than the sum of our parts". In software development terms, this gives us the ability to scale.
If this blog post has still not convinced or dissuaded you from building a Parallel Deepgram WebSocket Project to do real-time streaming, Check out our guide on using lower-level WebSockets with the Deepgram Streaming API for a Reference Implementation. This guide will help individuals duplicate the functionality that exists in the SDKs provided by and worked on in this community.
For those that want to join the community… reusable components, via software libraries and projects, have been a cornerstone and model of reuse and time savings. Why would we abandon that philosophy when a particular project gets us 80% of the way there? Being 80% closer to the goal is better than 0%; if something is missing in an open source project, build it and contribute it to the project. I am sure others in the community will be thankful that you did!
Sign up for Deepgram
Sign up for a Deepgram account and get $200 in Free Credit (up to 45,000 minutes), absolutely free. No credit card needed!
Learn more about Deepgram
We encourage you to explore Deepgram by checking out the following resources:
Unlock language AI at scale with an API call.
Get conversational intelligence with transcription and understanding on the world's best speech AI platform.