Ticket #136 (new defect)

Opened 1 year ago

Last modified 1 year ago

Cannot download feeds using HTTP chunked Transfer-Encoding

Reported by: trac-user Assigned to: fox
Priority: major Milestone:
Component: backend Version: 1.2.13
Keywords: Cc: blair@orcaware.com

Description

This is with 1.2.13. It appears that a feed served using chunked Transfer-Encoding cannot be handled by tts. The chunk transfer size ends up being parsed as part of the content, not the transfer.

$ wget -S http://blog.tmorris.net/feed/
--20:15:14--  http://blog.tmorris.net/feed/
           => `index.html'
Resolving blog.tmorris.net... 220.245.107.64
Connecting to blog.tmorris.net|220.245.107.64|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Sat, 11 Aug 2007 03:14:29 GMT
  Server: Apache/2.2.3 (Ubuntu) DAV/2 SVN/1.4.3 PHP/5.2.1 mod_ssl/2.2.3 OpenSSL/0.9.8c
  X-Powered-By: PHP/5.2.1
  X-Pingback: http://blog.tmorris.net/xmlrpc.php
  Last-Modified: Tue, 07 Aug 2007 05:21:28 GMT
  ETag: "b0b040ab319e9763065dc56e868b1c67"
  Status: 200 OK
  Keep-Alive: timeout=15, max=100
  Connection: Keep-Alive
  Transfer-Encoding: chunked

wget has an issue downloading this file, so you can see the chunk transfer size at the top of the file:

$ wget http://blog.tmorris.net/feed/
$ head -3 index.html 
3d22
<?xml version="1.0" encoding="UTF-8"?>
<!-- generator="wordpress/2.0.5" -->

Curl, Firefox and Safari have no issues downloading this URL, so it's a wget thing, but it lets you see the how the chunking works.

This is on an Ubuntu Dapper box using PHP 5.1.2.

Regards, Blair

Change History

08/11/07 10:21:48 changed by fox

tt-rss uses Magpie (which in turn uses Snoopy) to fetch data from the internet. If Snoopy can't understand chunked encoding, there is not much I can do - because of the way Snoopy is written it only makes my stupid head hurt when I try to figure something out in it.

Snoopy seems to be able to use Curl optionally for https, maybe it is possible to hack it so it would enable it on chunked encoding?