Why eDiscovery Uploads Get Super-Charged With 'Chunking'

10 March 2021 by Ross ediscovery

Takeaway: Chunking is the process of splitting a file into multiple parts (or ‘chunks’) that are uploaded simultaneously. It makes uploading lightning-quick and lets you resume broken downloads instead of having to start again from scratch.

With eDiscovery, you’re often dealing with large amounts of data. So, how do you get it into your eDiscovery software?

Most of the best eDiscovery applications run in the Cloud – a global network of servers that stores your data (apps like Dropbox and iTunes use the Cloud, too). A huge part of Cloud eDiscovery is uploading files into the Cloud as quickly as possible. So, how do next-generation eDiscovery applications offer lightning-fast uploads for hundreds of GBs, without you having to upgrade your hardware? They do it through ‘chunking’.

Chunking is an ingenious solution to our need for fast uploads. It’s a way of splitting up files into smaller chunks and uploading those chunks individually.

With regular uploads, you’re sending an entire file into the Cloud. With chunking, your eDiscovery software breaks up larger files into multiple parts (or ‘chunks’) and uploads these chunks separately. So, instead of uploading a large 1 GB file in a single shot, it’ll perhaps get broken into fifty 20 MB chunks. Once the chunks are on the Cloud server, they’ll get put back together to re-form the original file.

You’ll use the same internet bandwith with or without chunking, but behind-the-scenes technical factors make chunking quicker than regular uploads.

Many technical quirks make chunked uploads faster than non-chunked ones – even on a slow network. For example, a busy server will often prioritize chunked files because it assumes each chunk is coming from a different source. Whereas that same file, un-chunked, might get downgraded in the upload queue. Also, with some types of chunking (described a bit later), you’re uploading chunks to more than one server. So, even if one server pauses uploading a chunk, other servers keep uploading the remaining chunks.

Once your data is split into chunks, there are different ways it can get uploaded.

All uploads consist of these stages: connecting with a server, uploading the files and getting a confirmation that the upload was a success. But the mechanics of this process changes depending on the type of chunked upload you use.

  • Sequential uploading: Here, your software uploads the chunks in order. So for chunk number 2 to start uploading, chunk 1 has to finish and your eDiscovery application needs to get a confirmation of this. If a chunk fails to upload, your upload gets stalled.
  • Concurrent uploading: Here, your software doesn’t queue the chunks in any particular order. Rather, they all start uploading in parallel, as if they are completely different files. So, if chunk 1 fails, the software will retry the upload – but it’ll simultaneously keep uploading chunk 2, 3, 4, etc., despite chunk 1’s problems. One aspect of parallel uploads is that ‘more’ isn’t always ‘better’. There’s a sweet spot when it comes to the optimal size of each chunk, below which there’ll be too many chunks for the system to handle.
  • Multiple servers: In addition to uploading chunks in sequence or parallel, you can also include more than one server in the process. So, chunks 1, 9, and 15 might go to server 1, while chunks 2, 22, and 35 might go to server 2, and so on. This way, you spread the load between multiple servers, so it’s easier for them to handle the uploads. It does require a lot from the people running the servers, though. For example, they’ll need to maintain a dedicated database server and design their network with this sort of load sharing in mind.

Chunking might take a bit of extra work from your eDiscovery software, but it makes uploading so much easier for you.

We’ve seen how chunking makes uploading faster. But it also makes uploading less complicated because it limits the damage caused by upload errors. Ordinarily, you’d lose an entire upload if your computer hangs, your internet connection is abruptly cut off, or it takes too long for a server to respond. (You could have uploaded 920 MB of a 950 MB upload, and if your computer crashes, you’ll have to start again.) In contrast, the mechanics of chunking means that you can resume broken downloads, and even if there’s a problem uploading one chunk, the others get uploaded while your software tries to resolve the issue. This is perfect for busy attorneys who want to spend time building a case rather than solving tech hassles.

For chunking to work, though, you’ll need easy-to-use eDiscovery software.

The concept of chunking is pretty straightforward to understand, but the computer coding behind it is quite complex. So, you’ll need well-designed eDiscovery software and reliable tech support to handle upload issues that might pop up. We kept these two needs in mind when designing GoldFynch to leverage chunking, which is why it’s popular with our clients.

GoldFynch is an easy-to-use eDiscovery application that makes uploading quick and efficient. It also has a complete suite of the essential eDiscovery tools you’ll need.

Our eDiscovery service is perfect for small- and midsize law firms and companies.

  • It costs just $27 a month for a 3 GB case: That’s significantly less than most comparable software. With GoldFynch, you know what you’re paying for exactly – its pricing is simple and readily available on the website.
  • It’s easy to budget for. GoldFynch charges only for storage (processing is free). So, choose from a range of plans (3 GB to 150+ GB) and know up front how much you’ll be paying. It takes just a few clicks to move from one plan to another, and billing is prorated – so you’ll pay only for the time you spend on any given plan. With legacy software, pricing is much less predictable.
  • It’s simple to use. Many eDiscovery applications take hours to master. GoldFynch takes minutes. It handles a lot of complex processing in the background, but what you see is minimal and intuitive. Just drag-and-drop your files into GoldFynch and you’re good to go. Plus, it’s designed, developed, and run by the same team. So you get prompt and reliable tech support.
  • It keeps you flexible. To build a defensible case, you need to be able to add and delete files freely. Many applications charge to process each file you upload, so you’ll be reluctant to let your case organically shrink and grow. And this stifles you. With GoldFynch, you get unlimited processing for free. So, on a 3 GB plan, you could add and delete 5 GB of data at no extra cost – as long as there’s only 3 GB in your case at any point. And if you do cross 3 GB, your plan upgrades automatically and you’ll be charged for only the time spent on each plan. That’s the beauty of prorated pricing.
  • Access it from anywhere. And 24/7. All your files are backed up and secure in the Cloud.

Want to learn more about GoldFynch?