I have a REST API that accepts an Audio file via an HTTP Post. The API has support for Transfer-Encoding: chunked request header so that the file can be uploaded in pieces as it is being created from a recorder running on the client. This way the server can start processing the file as it arrives for improved performance. For example:
HTTP 1.1 POST .../v1/processAudio
Transfer-Encoding: chunked
[Chunk 1 256 Bytes] (server starts processing when arrives)
[Chunk 2 256 Bytes]
[Chunk 3 256 Bytes]
...
The audio files are typically short and are around 10K to 100K in size. I have C# and Java code that is working so I know that API works. However, I cannot seem to get the recording and upload working in a browser using javascript.
Here is my Test Code that does a POST to localhost with Transfer-Encoding:
<html>
<script type="text/javascript">
function streamUpload() {
var blob = new Blob(['GmnQPBU+nyRGER4JPAW4DjDQC19D']);
var xhr = new XMLHttpRequest();
// Add any event handlers here...
xhr.open('POST', '/', true);
xhr.setRequestHeader("Transfer-Encoding", "chunked");
xhr.send(blob);
}
</script>
<body>
<div id='demo'>Test Chunked Upload using XHR</div>
<button onclick="streamUpload()">Start Upload</button>
</body>
</html>I have a REST API that accepts an Audio file via an HTTP Post. The API has support for Transfer-Encoding: chunked request header so that the file can be uploaded in pieces as it is being created from a recorder running on the client. This way the server can start processing the file as it arrives for improved performance. For example:
HTTP 1.1 POST .../v1/processAudio
Transfer-Encoding: chunked
[Chunk 1 256 Bytes] (server starts processing when arrives)
[Chunk 2 256 Bytes]
[Chunk 3 256 Bytes]
...
The audio files are typically short and are around 10K to 100K in size. I have C# and Java code that is working so I know that API works. However, I cannot seem to get the recording and upload working in a browser using javascript.
Here is my Test Code that does a POST to localhost with Transfer-Encoding:
<html>
<script type="text/javascript">
function streamUpload() {
var blob = new Blob(['GmnQPBU+nyRGER4JPAW4DjDQC19D']);
var xhr = new XMLHttpRequest();
// Add any event handlers here...
xhr.open('POST', '/', true);
xhr.setRequestHeader("Transfer-Encoding", "chunked");
xhr.send(blob);
}
</script>
<body>
<div id='demo'>Test Chunked Upload using XHR</div>
<button onclick="streamUpload()">Start Upload</button>
</body>
</html>The problem is that i'm receiving the following Error in Chrome
Refused to set unsafe header "Transfer-Encoding"
streamUpload @ uploadTest.html:14 onclick @ uploadTest.html:24
After looking at XHR documentation i'm still confused because it does not talk about unsafe request headers. I'm wondering if its possible that XHR does not allow or implement Transfer-Encoding: chunked for HTTP POST?
I've looked at work arounds using multiple XHR.send() requests and WebSockets but both are undesirable because it will require significant changes to the server APIs which are already in place, simple, stable and working. The only issue is that we cannot seem to POST from a browser with psedo-streaming via Transfer-Encoding: chunked request header.
Any thoughts or advice would be very helpful.
Share Improve this question asked Jul 22, 2015 at 16:45 Mike KennewickMike Kennewick 1971 silver badge4 bronze badges 1- 2 You are not allowed to set that header. It's controlled by the user agent. Check the w3 spec for it here: w3/TR/XMLHttpRequest/#the-setrequestheader-method, specifically where it says: "The above headers are controlled by the user agent to let it control those aspects of transport". – Michael B. Commented Jul 22, 2015 at 16:56
2 Answers
Reset to default 3As was mentioned in a ment, you're not allowed to set that header as it's controlled by the user agent.
For the full set of headers, see 4.6.2 The setRequestHeader() method from W3C XMLHttpRequest Level 1 and note that Transfer-Encoding is one of the headers that are controlled by the user agent to let it control those aspects of transport.
- Accept-Charset
- Accept-Encoding
- Access-Control-Request-Headers
- Access-Control-Request-Method
- Connection
- Content-Length
- Cookie
- Cookie2
- Date
- DNT
- Expect
- Host
- Keep-Alive
- Origin
- Referer
- TE
- Trailer
- Transfer-Encoding
- Upgrade
- User-Agent
- Via
There is a similar list in the WhatWG Fetch API Living Standard. https://fetch.spec.whatwg/#terminology-headers
As other replies have already mentioned, you aren't allowed to set the "Transfer-Encoding" header yourself.
However, you also don't actually need to use HTTP chunked transfer encoding in order to incrementally stream a file to your server and start processing parts of it right away either. A regular HTTP POST works just fine for that. Even though it is transmitted as a single HTTP request, I believe the streaming/chunking magic happens for you at the TCP level (other people are wele to correct me if I'm wrong on where the magic specifically happens). I can confirm this works because I've done it with node.js and Express on the backend. I'm sure it probably works with other server side technologies as well.
HTTP chunked transfer encoding is only useful when you DON'T know the size of the stream you are going to be sending in advance (such as live video, video conference calls, remote desktop sessions, chats, etc.). And for these cases WebSockets are a more widely deployed solution that solve the same problem:
https://developer.mozilla/en-US/docs/Web/API/WebSockets_API
For your use case, where you DO know the size of the file in advance you are probably better off sticking to your XmlHttpRequest and abandoning the chunked transfer encoding. Alternatively, you can give the new Fetch API a try:
https://developer.mozilla/en-US/docs/Web/API/Fetch_API