Commit 74058d1

committed

Limit concurrent requests and set the download delay.

1 parent 569d3c3 commit 74058d1Copy full SHA for 74058d1

File tree

-2

lines changed

-2

lines changed

Lines changed: 2 additions & 2 deletions

Original file line number	Diff line number	Diff line change
`@@ -21,12 +21,12 @@`
`21`	`21`	`ROBOTSTXT_OBEY = True`
`22`	`22`
`23`	`23`	`# Configure maximum concurrent requests performed by Scrapy (default: 16)`
`24`		`-#CONCURRENT_REQUESTS = 32`
	`24`	`+CONCURRENT_REQUESTS = 1`
`25`	`25`
`26`	`26`	`# Configure a delay for requests for the same website (default: 0)`
`27`	`27`	`# See http://scrapy.readthedocs.org/en/latest/topics/settings.html#download-delay`
`28`	`28`	`# See also autothrottle settings and docs`
`29`		`-#DOWNLOAD_DELAY = 3`
	`29`	`+DOWNLOAD_DELAY = 5`
`30`	`30`	`# The download delay setting will honor only one of:`
`31`	`31`	`#CONCURRENT_REQUESTS_PER_DOMAIN = 16`
`32`	`32`	`#CONCURRENT_REQUESTS_PER_IP = 16`

Comments

(0)