Caching

[René Neumann]

Am 26.04.2013 15:47, schrieb Jason A. Donenfeld:
> On Thu, Apr 18, 2013 at 10:03 PM, René Neumann <lists at necoro.eu> wrote:
>>
>> Fortunately a couple of servers implement the X-Sendfile extension (or
>> synonyms thereof) that allow to pass a path to the webserver which then
>> serves it directly like every other static file.
> 
> 
> I would be potentially interested in implementing this as we move
> closer to having FastCGI/SCGI support (which is a bit of a ways away
> due to memory management strategies). Personally, I'm more interested
> in nginx's X-Accel-Redirect than X-Sendfile, but supporting either
> isn't too difficult. Remind me of this in a few months.
> 
I'm still not sure this is possible at all w/o having a complete new
caching algorithm, as you can't know whether the URI/path you've sent to
the webserver to serve still contains the content you wanted to send the
moment the web server processes it. There is also the chance that the
server only sees a partial written file.

Main problem is the missing ack in the protocols. Though perhaps a
reasonable time period where writes are not allowed (5 mins?) could be
sufficient.

- René

P.S.: While writing this email, I had an idea for an approach using this
timeout method:

1.) On request, copy the content part of the file to an extra
xsendfile-file (with 'sendfile()' the overhead should be reasonable).
2.) Send back the path to this new file.

*.) On each request check each file in the xsendfile-dir and remove it
if older than 5? minutes.

The part of "*" could also be taken by a cronscript or sth alike.

Also one should take care to not create the same content trillions of
times, so a check should be added, that does not create the
sendfile-file again if the cache did not change in the meantime.

If noone objects I'll see if I can give it a try during the weekend... :)