No More Zapier Timeouts: Transcribe Large Audio Files with Deepgram and Zapier’s Webhooks

Deepgram recently became available in Zapier, making it a breeze to transcribe audio without writing any code. Check out this docs article, “Zapier and Deepgram” to get started creating your first zap with our new Zapier integration.

While creating a basic Zap with Zapier and Deepgram is a straightforward process, some users will hit a wall when they find out that Zapier has a thirty second timeout on all actions that run within its workflows. Deepgram is very fast at transcribing large files, but you might find that files larger than 200MB sometimes take longer than the 30 seconds allowed by Zapier.

The good news is that there is a workaround to this problem, although it isn’t a no-code solution. You will have to create a basic server to handle an HTTP request. But don’t worry - this tutorial will walk you through how to build the entire workflow.

Overview

We’ll be using these tools and integrations to build our workflow:

• Deepgram

• Zapier

• Node Express

• A deployment service of your choice to deploy the Node server (I used Fly.io)

• Amazon S3

• CloudConvert

• Dropbox

Here is a high-level overview of what we’ll build. It’s important to understand that this workflow actually requires the building of two separate Zaps.

Step One: Build the Server

We will start by building a server, since we will need this for both the Zaps we are going to create in Zapier. We’ll build a Node Express server, but you can easily do this in your language of choice.

Step Two: Create Zap #1

We will create a Zap to transcribe an audio file with Deepgram. The audio file will come from an S3 bucket, and then we’ll use CloudConvert to create a signed URL of the audio file. The audio file will be sent to Deepgram along with a callback URL (for the Node server we built) where we want the transcription response to be sent.

Step Three: Create Zap #2

We will create a Zap that uses Zapier webhooks to watch for data sent to a Zapier webhook URL. That webhook URL will receive the transcript sent from our Node server. This Zap will trigger once the transcription shows up, and then the Zap will create a text file of the transcription in Dropbox.

Walkthrough

Step One: Build the Server

Deepgram’s callback feature lets you provide a callback URL for where you would like the transcription to be sent once it is ready. This is a helpful feature when transcribing larger files that might take longer to transcribe (which means this is a good work-around for dealing with Zapier’s timeout restriction).

If you would like to understand the callback feature in more depth, I recommend you read this article in the Deepgram docs.

Node Express Server

Here is the starting code for a basic Node server. (The example project can be found in this github repo.) This code example is missing logic within the /hook endpoint and the sendTranscriptiontoZapier function that we will add soon.

Note: we include Axios because we will eventually be using that library to post the transcription data to the Zapier webhook URL.

POST /hook Endpoint

Inside the /hook endpoint, we need to add some logic to check if the transcript has arrived. If it has arrived, we send it to Zapier. 

sendTranscriptiontoZapier function

We don’t have the webhook URL from Zapier yet, but we can write the logic to make the post request (later on we’ll put in the correct webhook URL). Use the command `npm install axios` to add Axios, which is a library that makes it easier to write the POST request.

Deploy the server

You will need to deploy the server. You can do this using your service of choice. I used Fly.io. They provide documentation for deploying Javascript apps, but I found this YouTube video and this article by Hayk Simonyan to be all I needed. You may also want to look at the configuration files in my github project to see how I configured my project for Fly.io.

Step Two: Create Zap #1

Now we will create the first Zap. This Zap will trigger when we add an audio file to a specific folder in an S3 bucket. Let’s take it step by step through the Zapier form.

Trigger: New File in Folder in Dropbox

In the dashboard in your Zapier account, click on the “Create Zap” button in the left nav. This will set up a template with a trigger and an action.

For the “App & event”, choose Amazon S3 as the app and “New or Updated File” as the event. Then connect your AWS account by providing your access key and secret access key.

In the “Trigger” section, select a folder from your Dropbox account that you want Zapier to connect to. Zapier will watch that folder, and when a file is added to it, the workflow will run.

Test the trigger. This will reveal the files that are in the Dropbox folder. If you don’t have any files in there, you need to add an audio file so we can test the workflow.

Select one of the “records” (audio files) and click “Continue with selected record”.

Note: Some apps put limits on the file size that can be used in their Zapier integration. In my testing, S3 was the best integration to use because I never hit a limit.

Action: Convert a File

We’ll use CloudConvert to turn the audio files into a signed URL, the format that the Deepgram integration can accept.

For the “App & event”, choose CloudConvert as the app and “Convert a File” as the event. Then connect to a CloudConvert account by adding your API key (if you don’t have one, you will have to sign up for one at cloudconvert.com).

Add the file and choose an output format and input format. It doesn’t matter what you choose for the output format, as long as it is an audio format that Deepgram will accept. (See the list here). We aren’t actually doing this because we want to convert the file - we are using CloudConvert to create a URL of the temporarily stored file, which it does for all the files it converts.

Test this step. Notice that we get a URL returned as the “Task Result Files.” We’ll use this to send the file to Deepgram in the next action.

Action: Create Transcription (Callback)

For the “App & event”, choose Deepgram as the app and “Create Transcription (Callback)” as the event. Then connect to your Deepgram account by adding your API key (if you don’t have one, you will have to sign up for one at deepgram.com).

Add the URL for your deployed Node server. Be sure to include the endpoint route ‘/hook’ at the end of the URL. For the Audio URL, select the Direct Media Link from the Dropbox record.

Click “Test step”. You should see a response that includes a Request ID.

Step Three: Create Zap #2

Now we will create another separate zap. Go back to the dashboard and click on “Create Zap.”

Trigger: Catch Hook in Webhooks

For the “App & event”, choose Webhooks by Zapier as the App and Catch Hook as the event.

In the “Trigger” section, we should only add a Child Key if we want to limit what we get back from the response. I prefer to leave it blank so that we get back all the metadata, since the metadata is useful for naming the transcription file.

In the “Test” section, we are provided with the webhook URL.

We need to add this to the Node server in this line where it says “WEBHOOK_URL_HERE”

Then redeploy the server. You will need to run Zap #1 again so that data gets sent to the webhook URL. After you run Zap #1 again and see a Request ID response, you can test this trigger step in Zap #2.

If everything worked correctly, you should see a new record appear. This is the data that was sent to the Zapier webhook URL. It includes the transcript, along with everything else in the response data.

Select the record, then click “Continue with selected record.”

Action: Create Text File in Dropbox

The last thing we need to do is create a text file of the transcript. We will do this by setting up the Zap to add a text file to a specific folder in Dropbox when the transcript is received.

For the “App & event”, choose Dropbox for the app and Create Text File as the event. Then connect your Dropbox account.

In the “Action” section, choose the folder you want the transcript to be put into. You must select a File Name. I prefer to use the request id from the metadata, because that will give a unique name for each transcript. Then select “Results Channels Alternatives Transcript” for the File Content. This is the text transcript.

Run the test. You can now head over to Dropbox to see the transcript text file!

Conclusion

We’ve successfully come up with a work-around to Zapier’s pesky thirty-second timeout. This will allow us to transcribe longer audio files with Zapier and Deepgram (although be aware that some Zapier integrations such as Dropbox put their own limits on file sizes, which is why I recommend using S3 as your storage provider).

We also have a better understanding of Deepgram’s callback feature and Zapier’s webhooks. I hope this information will be a starting point for you to start building more automations with Deepgram and Zapier!

Sign up for Deepgram

Sign up for a Deepgram account and get $200 in Free Credit (up to 45,000 minutes), absolutely free. No credit card needed!

Learn more about Deepgram

We encourage you to explore Deepgram by checking out the following resources:

1. Deepgram API Playground 

2. Deepgram Documentation

3. Deepgram Starter Apps