Zencoder benchmarked: 2x-10x faster than the competition
Speed has been part of Zencoder’s DNA ever since the first line of code was written. Having been in the video transcoding industry for a few years, we knew that speed was important to most customers. UGC sites don’t want to make users wait for video. Professional publishers want to make sure that they can get content online as quickly as possible. And broadcasters want to publish breaking news immediately – one of the reasons that PBS moved to Zencoder. So that’s why Zencoder is architected for speed.
We’ve known for a long time that Zencoder is fast – we’ve optimized every corner of our system for speed, and anecdotally, customers tell us that we beat their previous on-premise systems, or competitive cloud services, by a wide margin.
But we haven’t really known how fast. So we decided to find a reputable third party to do a performance benchmark of cloud-based transcoding providers. CloudHarmony came quickly to mind. We’d seen their previous work benchmarking cloud computing servers and CDNs, and they seemed like a good fit. We reached out to them, and (full disclosure) Zencoder sponsored a project to benchmark leading cloud-based encoders.
We wanted the test to be as fair and transparent as possible. You’ve seen enough benchmarks to be suspicious of biased or partial tests, right? (If AT&T is really the best 3G network in the country, why do my calls keep dropping?) So CloudHarmony picked four representative files to test: an HD file, a long-form SD file, a short-form SD file, and an audio file. They encoded these files to the exact same settings on each service, encoded them simultaneously on each service, and cycled the tests throughout a 24-hour day across seven days to eliminate time-of-day or day-of-week differences. The methodology is entirely transparent. If there are any flaws in the methodology, we’d be happy to have them re-run the tests.
The results: Zencoder is 2x-5x faster than competing services on average. With HD content and long-form content, we’re up to 10x faster. We’re the only provider in the test that encoded HD content faster than realtime. And the advantages were consistent – on nearly every test, we come out on top.
The full results are available at CloudHarmony.
This test doesn’t take scalability into account. Scalability and burstability are one of the biggest advantages of the cloud – cloud-based infrastructure like EC2 or Zencoder lets you transparently burst from a small number of resources to a large number of resources. The problem is that not every provider handles this in the same way. With Sorenson and Panda, you have to manually provision new servers when you need them, which means that bursting is your responsibility. Zencoder and Encoding.com take care of this for you. We’re big believers in our model – customers should be able to send us one video one day, and 10,000 videos another day, and have the 10,000 videos automatically processed just as quickly as the single video. But because of the different models used by different cloud encoding vendors, it’s difficult to determine an apples-to-apples comparison across each model. So that scalability benchmark will need to wait for another day.
Similarly, we tried to make transfer time a non-factor by using Amazon S3 as file storage. Every transcoding provider in the test uses Amazon EC2 for transcoding, and we stored files in the same region (US-East) in order to eliminate variability based on transfer time. Transfer times to the cloud are important, but that too will need to wait until another test.
So, what now?
Zencoder is architected for speed, and the results prove this. But to be honest, we aren’t perfect. We do some things right: well-tuned encoding software, intelligent queuing algorithms, fast hardware, accelerated file transfer. But we also know where we can improve. There are optimizations we can make, small and large, that will further improve our speed, and they’re all on our roadmap. We’re working on things like Aspera integration, eliminating bottlenecks on the transcoding process, faster servers, smarter job distribution, and lower queue times. Plus the ever present work of finding and eliminating unnecessary overhead in our system.
This means that the next time we publish speed benchmarks, we’re less interested in comparing our speed against our competitors, and more interested in comparing speed against Zencoder (in previous versions). So if we haven’t announced further speed increases by (say) Q1 of next year, get in touch and lodge a formal complaint.
The big picture
The Y Combinator motto is “Make something people want.” Through conversations with hundreds of video publishers, we discovered that people want fast transcoding times. But that isn’t all they want. Video publishers also want things like reliability, high scalability, awesome video and audio quality, and easy integration. So raw speed is only one aspect of performance, and at Zencoder, we’re committed to the highest performance in every area. We’d love to be able to objectively quantify our performance in these other areas as well, so look for more benchmarks over the coming months.
Appendix: help us out!
If you like what we do and believe in our mission of bringing the best possible video performance to the cloud, vote for Zencoder in the Streaming Media Reader’s Choice awards! Voting closes Friday, and it just takes a few seconds. (You don’t have to vote for every category.) We’d really appreciate your vote.
Why would you perform a “HD” test with a 250MB source file?, that is not representative of the real challenge for content owners moving to the cloud.
Hi Jack. Just so I understand, are you saying that the real challenge is the transfer of large files into the cloud? I think this test is significant enough to extrapolate roundtrip times on much larger source files.
The main challenge for content producers to move from on premise to cloud encoding is transfer time of mezzanine files to the cloud. You chose a test criteria (245MB source on S3) which conveniently downplayed your competitors main speed advantage over you, Instant Encoding from Encoding.com. If you tested with 10GB source file on public FTP instead of S3, Encoding.com would be magnitudes faster on overall job time (transfer, processing, upload) What do you think we all have our ProRes source already sitting on S3 or something?
Thanks. I understand your point better now. You might want to head over to the Cloud Harmony blog post and read Jon’s response to a similar question in the comments. Bottom-line is that in this test we were trying to isolate encoding times and limit the effects of public internet transfer variability. Jon mentions above that we will be running more benchmarks to cover things like scalability, transfer speeds, quality control, advanced video features, etc.
To respond more directly about FTP transfer speeds, Jon put it well in the comment over at CH:
“At Zencoder, the average download bandwidth via FTP (not S3) is 80Mbps. At 80Mbps, the HD job would take 21 seconds longer for Zencoder to process and 0 seconds longer for Encoding.com, so times would be 176 seconds at Zencoder and 238 at Encoding.com. Once bandwidth drops to 20Mbps or so, we’re tied, and at lower bitrates, we’re affected equally.”
Jon is absolutely correct in the context of your test with small source files on varying FTP connection speeds, but the delta becomes very significant with large source files.
Hi Jack – the thing to keep in mind is that the max speedup you can get with a feature like Instant Encoding is the lower of: (1) the file transfer time, or (2) the first-pass encode time. First-pass encode time is typically about 1/3 of total transcoding time, so given a large source file and/or a slow pipe, the max speed advantage would be about 1/3 of actual transcoding time. In the benchmarks done, Zencoder’s encode time is >= 50% faster than Encoding.com’s in almost every case, so even in the worst possible case, Zencoder still ends up being faster.
Ah ok you are assuming 2 pass, we think that x264 does a pretty amazing job with 2 pass and hardly ever use it.
This is a well informative content.The performance leader in cloud-based video encoding.The written skill is so good.I am very impress to this content.This is an amazing.I appreciate to this one.Thanks to share this blog with us.
how do your speeds compare with eTransmittal or some of their FTP alternatives for secure file transfer?
Thanks so much for this. This is awesome post I ever seen on internet. This is rare to find that’s why difficult to understand. Anyway, you are definitely someone that has something to say that people need to hear. Keep up the good work. Keep on inspiring the people.
For more information regards:
IT Service Provider Benchmarking
Thanks so much for this. This is awesome post I ever seen on internet. This is rare to find that’s why difficult to understand. Anyway, you are definitely someone that has something to say that people need to hear. Keep up the good work. Keep on inspiring the people.
For more information regards:
IT Service Provider Benchmarking
I
appreciate the information provided by you. This is interested to
know about Zencoder benchmark.
If you want to know more about IT Service Provider.
go to the link below. IT Service
Provider Benchmarking