One of the challenges of running an Internet-based business is that there are no geographical limits to where your customers come from. As of this writing, we have customers from every continent on Earth except Antarctica. While the Internet is a great enabling technology in that regard, there are physical limits. The biggest problem: Light isn’t fast enough.
Yes, you read that correctly. Light and electricity travel at roughly the same speed, yet sending either one 10’s of 1,000’s of miles through multiple routers takes an easily measurable and equally disappointing amount of time. It’s this length of time, “latency” being the buzzword, that makes Internet communication over great distances sluggish. The communication protocols used over the Internet (namely TCP is of interest for this article) must send a packet from source to destination and wait for acknowledgement of receipt. While this mechanism is reliable, it takes hundreds of milliseconds for a packet to be sent around the world and the acknowledgement to return. This constant game of
“I sent it, did you get it?” > “Yeah, I got it, proceed.” > “OK, here’s another…let me know when you get it.”
exposes latency as the killer of throughput for a TCP connection. “So,” you say, “what the heck does this have to do with a render farm service, anyway?” Glad you asked!
Using our agent, you upload scene files and all their external assets to our farm. Our agent downloads results as they are rendered. What if your scene file is multiple Gigabytes in size? That can take literally hours to upload if you are on the other side of the planet from us. Same problem if you have a couple thousand frames of 4k images to download. The good news is, there is an answer for this. The even better news is that we have just implemented it in our Mac and Windows rendering agents.
Our agents now transfer all data, both up and down, using multiple simultaneous TCP sessions. Using 4 or more sessions, they move files in “chunks”, with each session moving their chunks of the file. This means that various parts of the same file are being transferred over the wire simultaneously, with each individual TCP session capable of whatever the Internet between you and us will allow. The data transfer looks more like what Bitorrent and other P2P technologies do, but we do it all from one machine to another rather than from many to one.
Let’s say you’re one of our clients in Rome, Italy, and you have a 10Mbps symmetric Internet connection. Since you’re so far away from our farm from an “Internet distance” perspective, you may normally only be able to upload to and download from our farm at 2Mbps. If your agent, however, uses 5 simultaneous sessions to connect to our farm and move data, it can now move an aggregate of 10Mbps up and down. Latency be darned, darned to heck!
This, in essence, makes our farm transfer data as though it is directly connected to your ISP, with the only speed limit being the bandwidth limits on your Internet connection. If you have a big pipe, we can fill it. This means faster results to you, since job uploads are faster and rendered result downloads are faster. It’s all part of the entire workflow process we are seeking to improve for our customers, who see Pixel Plow as a valuable and efficient alternative to creating and operating their own render farm. Pixel Plow has closed the Internet latency gap to enable their render farm customers around the world to operate on par with each other.
Multi-session data transfers, just another way Pixel Plow is helping you to Render Endlessly…
