Http 429/Too many requests

Support requests, bug reports, etc. go here. Dedicated servers / VDS hosting only
erw
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 04 Mar 2017, 17:19

Http 429/Too many requests

Postby erw » 04 Mar 2017, 17:24

Hi,

Recently I set up tt-rss and everything goes fine except one thing:
I need to fetch feeds from one website but couple different RSS and sometimes there are HTTP 429(too many requests) for some feeds.

Is there some kind of workaround?
Last edited by erw on 05 Mar 2017, 00:23, edited 1 time in total.

User avatar
sleeper_service
Bear Rating Overlord
Bear Rating Overlord
Posts: 884
Joined: 30 Mar 2013, 23:50
Location: Dallas, Texas

Re: Http 429/Too many requests

Postby sleeper_service » 04 Mar 2017, 21:20

seriously?

how about not fetching them as often?

erw
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 04 Mar 2017, 17:19

Re: Http 429/Too many requests

Postby erw » 04 Mar 2017, 21:45

So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.

linoth
Bear Rating Trainee
Bear Rating Trainee
Posts: 22
Joined: 15 May 2013, 11:34

Re: Http 429/Too many requests

Postby linoth » 04 Mar 2017, 22:20

Well, since you've supplied literally zero useful details on the actual problem, it's hard to offer a given solution.

If it's something like fetching every four hours, you could try to stagger them by hand, by disabling updates on some and manually re-enabling an hour later, or what-have-you, in order to satisfy the provider's rate limit. That should work indefinitely unless you're that close to the limit, but if your server ever goes down for an extended period and can't update, you'll have to set it up all over again.

You could also potentially dig around and try to change the internal update scheduling, or set up a cron job to manually run the update script every hour, or however long you needed to wait between fetches, in order to enforce your own rate limit that way. Alternatively, you could lower the number of feeds that could be updated per daemon execution, if we're talking an X requests in Y minutes sort of thing.

I don't believe plugins would be of much use here, but I could very well be wrong.

I suppose you could get really fancy and write your own proxy script of sorts, which kept track of your own requests and enforced a rate limit before allowing the update to proceed as scheduled. That's starting to get a little excessive and convoluted, though.

But these are all pretty much common sense things.

Regardless, I'd recommend mellowing the hell out and actually providing some useful information before you get your ass banned. You're the one not being remotely helpful here. We don't know the site in question to get any answers for ourselves, we don't know the rate limit you're trying to conform to, we don't know how often you're actually updating, we don't know the actual scope of how many feeds we're talking about here, we don't know a goddamn thing and you're expecting miracles to come out of our asses to satisfy your entitlement.

erw
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 04 Mar 2017, 17:19

Re: Http 429/Too many requests

Postby erw » 04 Mar 2017, 22:26

That what I thought.

It looks like there is quite enough info to answer the question:
1 provider, multiple feeds - HTTP 429 in result. What else do you need? How does it matter what provider is it or what the limit is?
Problem is clear - too many requests since tt updates them very quickly without pause between requests.

I hope simple sleep should help.

JustAMacUser
Bear Rating Overlord
Bear Rating Overlord
Posts: 373
Joined: 20 Aug 2013, 23:13

Re: Http 429/Too many requests

Postby JustAMacUser » 04 Mar 2017, 22:49

@erw,

You're so pleasant in your responses. It's really quite wonderful. /s

erw wrote:So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.


[emphasis mine]

Side note: It's nice that you start your journey in the forums being snarky to a moderator.

So, you already "mentioned" that "it's not often"? Let's take a look-see:

erw wrote:Hi,

Recently I set up tt-rss and everything goes fine except one thing:
I need to fetch feeds from one website but couple different RSS and sometimes there are HTTP 429(too many requests) for some feeds.

Is there some kind of workaround?


At what point in your post did you actually mention that you're not fetching often?

At any rate, what is "often"? For some that might be 15-minutes and others that might be not more than once a day.

Frankly, your attitude to those helping you is less than desirable but I'm a nice guy so I'll give you some advice.

erw wrote:How does it matter what provider is it or what the limit is?


It matters because if we know who the provider is, we might be able to give some better advice. Larger ones like FeedBurner have limits that others have already encountered and managed; if that was the provider, we could give more targeted advice. If it's joe-some-guy-blog-with-feeds-for-each-category that's a different story.

erw wrote:Problem is clear - too many requests since tt updates them very quickly without pause between requests.


The problem really is clear. Abundantly so. You're fetching too frequently, as you said. The solution is to fetch less frequently.

That being said, you could write a plugin to hook the maintenance function in TT-RSS. It runs hourly by default, and then you could manually modify fetch times for the feeds based on when other feeds were updated. Or you could hook just before feeds are fetched, check the times and return erroneous data to TT-RSS so it aborts fetching, etc. Both are hacks but they could be coded in plugins to do what you want without much upkeep thereafter. Another thing you could do if you're on a VPS is add more IPs and setup some scripts to rotate which IP is used for outbound requests to port 80 and 443.

Do you want the real solution?

Talk to the owner of the web site. Explain what you're doing and why, and ask how often he or she recommends you fetch. Respect their response. Problem. Solved.

User avatar
sleeper_service
Bear Rating Overlord
Bear Rating Overlord
Posts: 884
Joined: 30 Mar 2013, 23:50
Location: Dallas, Texas

Re: Http 429/Too many requests

Postby sleeper_service » 04 Mar 2017, 22:54

erw wrote:So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.

not my fault if you're too mentally insufficient to understand that if you're hammering on ONE provider with LOTS of feeds, they might very well decide you're trying to... what was it ??? "Http 429/Too many requests" make ... "too many requests".

but, hey, if you're too dumb to understand that, I'd be happy to suspend your account for a couple weeks so you can try to think on it some more.

just let me know.

e: on second thought, why muffle your shit when you're providing so much mock worthy fodder.

erw
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 04 Mar 2017, 17:19

Re: Http 429/Too many requests

Postby erw » 04 Mar 2017, 22:57

Like I care.

erw
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 04 Mar 2017, 17:19

Re: Http 429/Too many requests

Postby erw » 04 Mar 2017, 23:12

C'mon guys, I expected like 2-3 more.
But I should admit @JustAMacUser - that was incredible, like lots of words, really lots.

User avatar
sleeper_service
Bear Rating Overlord
Bear Rating Overlord
Posts: 884
Joined: 30 Mar 2013, 23:50
Location: Dallas, Texas

Re: Http 429/Too many requests

Postby sleeper_service » 04 Mar 2017, 23:51

I'm torn, ban him and delete the thread, or move it to the comedy bistro?

JustAMacUser
Bear Rating Overlord
Bear Rating Overlord
Posts: 373
Joined: 20 Aug 2013, 23:13

Re: Http 429/Too many requests

Postby JustAMacUser » 05 Mar 2017, 00:11

sleeper_service wrote:I'm torn, ban him and delete the thread, or move it to the comedy bistro?


Well, I don't think this place is a democracy. So...

e: If it were my choice, I'd vote is comedy bistro..

User avatar
sleeper_service
Bear Rating Overlord
Bear Rating Overlord
Posts: 884
Joined: 30 Mar 2013, 23:50
Location: Dallas, Texas

Re: Http 429/Too many requests

Postby sleeper_service » 05 Mar 2017, 01:17

JustAMacUser wrote:
sleeper_service wrote:I'm torn, ban him and delete the thread, or move it to the comedy bistro?


Well, I don't think this place is a democracy. So...

e: If it were my choice, I'd vote is comedy bistro..


I'll don't have the cosmic godlike powers to actually move it into the 'bistro, we'll have to wait for his nibs to surface ;)

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Http 429/Too many requests

Postby fox » 05 Mar 2017, 09:17

this sure was a thread. idk about its bistro worthiness though.

also not sure why op got all salty re: his obviously torrent feeds *shocked gasps in the audience*.

technically i guess in this situation there's no provided solution. other than running some kind of adhoc proxy like suggested above. i'm not sure if something like that could be implemented with the way updating works and whether it would be a good idea.

disconn3ct
Bear Rating Trainee
Bear Rating Trainee
Posts: 37
Joined: 12 Apr 2013, 15:07

Re: Http 429/Too many requests

Postby disconn3ct » 07 Mar 2017, 06:13

fox wrote:this sure was a thread. idk about its bistro worthiness though.

also not sure why op got all salty re: his obviously torrent feeds *shocked gasps in the audience*.


Could be hammering reddit porn.

User avatar
sleeper_service
Bear Rating Overlord
Bear Rating Overlord
Posts: 884
Joined: 30 Mar 2013, 23:50
Location: Dallas, Texas

Re: Http 429/Too many requests

Postby sleeper_service » 07 Mar 2017, 07:21

probably nothing that high class.


Return to “Support”

Who is online

Users browsing this forum: No registered users and 8 guests