Missing < > & characters from article content

[adq]

This isn't a support request, just a problem I've tracked down. I updated my server recently and suddenly noticed that tt-rss was stripping html characters from feeds seemingly at random. A bit of hunting later showed it was a problem with libxml2 and php. The bug here http://bugs.php.net/bug.php?id=45996 describes exactly what I am seeing.

I downgraded my server to libxml2 2.6.32 and it started working properly again.

I also tried upping to libxml2 2.7.1 and rebuilding php against that (in case it had some compile-time detection logic), but the problem recurred.

I'm happy with an older version of libxml2, and obviously this isn't a tt-rss issue; just thought I'd mention it in case someone else has the same problem.

[fox]

Eh, I never encountered this problem. Thanks for reporting!

[Qwark]

I have (had) the same problem as well.

I modified simplepie.inc to replace the problematic chars to their numeric equivalents just before parsing. the XML parser leaves them alone and everything works again as expected (for me)

(remember to set ENABLE_SIMPLEPIE to true in config.php to make ttrss actually use simplepie) But I guess this works just as well in magpie if you paste these 3 lines at the right place.

Code: Select all

diff -Naur --exclude .backups --exclude icons --exclude '*.png*' tt-rss-20080919.org/simplepie/simplepie.inc tt-rss-20080919.alex/simplepie/simplepie.inc
--- tt-rss-20080919.org/simplepie/simplepie.inc 2008-09-19 02:00:03.000000000 +0200
+++ tt-rss-20080919.alex/simplepie/simplepie.inc    2008-09-21 01:29:13.000000000 +0200
@@ -12761,6 +12761,10 @@
        xml_set_character_data_handler($xml, 'cdata');
        xml_set_element_handler($xml, 'tag_open', 'tag_close');

+       $data=str_replace("<","<",$data);
+       $data=str_replace(">",">",$data);
+       $data=str_replace("&","&",$data);
+
        // Parse!
        if (!xml_parse($xml, $data, true))
        {


[thecount]

Same problem, Qwark's fix works for me.

[fox]

I'm not sure I can merge the fix into trunk, though - I think it might break things for those with non-broken libxml.

I'll keep this thread sticky as a reference for people whose system experience this problem.

Edit: The patch above also solves the problem with missing & (&amp;) in article links.

[padde]

Same problem here... php 5.2.6 + libxml2 2.7.2 (Gentoo).

[candrews]

I didn't notice this thread, so I reported a bug here: http://tt-rss.org/trac/ticket/224

[fox]

I've added the entry to the FAQ which links to this thread and the ticket you created.

[padde]

Umm, could somebody provide a patch for magpie?

I'm trying to package tt-rss for Gentoo, but this is a showstopper (as the bug shows up with the versions of php/libxml2 that are being shipped with Gentoo currently). Until the problems in php/libxml2/wherever are fixed, I'll automatically apply the patch(es) during installation to provide a working tt-rss to Gentoo users.

[fox]

Try adding the same three str_replace() calls after magpierss/rss_parse.inc:158, e.g.

Code: Select all

     xml_set_character_data_handler( $this->parser, 'feed_cdata' );

    // add these three lines
    $source=str_replace("<","<",$source);
    $source=str_replace(">",">",$source);
    $source=str_replace("&","&",$source);
   
    xml_parse(), etc
    ...


[padde]

Great, that worked. Thanks

I attached the two patches (in -Nur format).

[fox]

I'll try to check how unbroken libxml operates with those tomorrow.

[paulproteus]

Howdy Fox,

Any updates on this?

It seems to me that the suggested patches should have zero impact on a working libxml2.

[fox]

Oh crap, I forgot all about it. I'll merge the patches into trunk and see whether it breaks stuff for me.

[Bernd]

There seems to be some more problems with spezial characters within (image-)links.

The source

Code: Select all

<p><img class="alignnone size-full wp-image-957" title="bestandene Diplompr&#252;fungen an Fachhochschulen" src="http://blog.bernd-distler.net/wp-content/uploads/2008/12/vdi_diplompruefungen.png" alt="" width="480" height="320" /></p>


becomes

umlauts.png (4.09 KiB) Viewed 1978 times

I have this problem with diferent feeds, not only with my one