If you upload language files to WebTranslateIt, you may have noticed that a big file could take a couple of minutes to import. Reading the file was never the slow part. Even large files parse in under a second. The time went into writing each segment to the database, one after the other, with more checks per segment than were really needed.
We dug into it and found that the slow part didn’t just take long, it got slower as your files grew. Worse, re-uploading a file you had already imported, with only a handful of changes, took longer than importing it the first time. Since re-uploading is what most of you do every day, that was exactly the wrong way round.
That’s fixed. Here is what we measured on a file with 4,000 segments:
| Before | After | |
|---|---|---|
| Importing the file for the first time | 51 s | 33 s |
| Re-uploading it with 10% of the texts changed | 66 s | 9 s |
Re-uploads are now about seven times faster, and first imports about a third faster. The bigger the file, the bigger the gain, because the cost no longer grows with the size of the file.
Nothing changes on your side. It applies to every file format we support, to uploads through the web interface, the API, the command-line tool and the integrations, and it doesn’t alter what an import does: the same segments are created, updated, flagged or made obsolete as before.
This is the first of a series of improvements to file imports. The next steps will make re-uploading unchanged files close to instant, and speed up first-time imports of very large files further. We’ll share numbers as we go.
As always, if you notice anything unusual after an import, let us know.