• # Problème avec python 3.7?

    Posté par . En réponse à la dépêche Crowl, un crawler SEO open-source. Évalué à 1. Dernière modification le 03 septembre 2018 à 19:22.

    Bonjour,

    Je suis assez intéréssé par votre project et j'aimerai bien contribuer, cependant je n'arrive pas a le faire tourner.

    A l'installation des dépendances python, reppy 0.4.9 génère des erreurs lors de la compilation(gcc 8.2).
    J'ai donc installé la dernière version à la place.

    Et au lancement de crowl, scrappy démarre bien mais twisted n'a pas l'air content du tout!

    $python crowl.py -l -c -u https://www.algoo.fr/ -b algoo
2018-09-03 17:34:12 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: scrapybot)
2018-09-03 17:34:12 [scrapy.utils.log] INFO: Versions: lxml 4.2.1.0, libxml2 2.9.8, cssselect 1.0.3, parsel 1.4.0, w3lib 1.19.0, Twisted 18.4.0, Python 3.7.0 (default, Jul 15 2018, 10:44:58) - [GCC 8.1.1 20180531], pyOpenSSL 18.0.0 (OpenSSL 1.1.0h 27 Mar 2018), cryptography 2.2.2, Platform Linux-4.14.67-1-MANJARO-x86_64-with-arch-Manjaro-Linux
2018-09-03 17:34:12 [scrapy.crawler] INFO: Overridden settings: {'CONCURRENT_REQUESTS': 5, 'DEPTH_PRIORITY': 1, 'DOWNLOAD_DELAY': 0.5, 'LOG_LEVEL': 'INFO', 'RETRY_ENABLED': False, 'ROBOTSTXT_OBEY': True, 'SCHEDULER_DISK_QUEUE': 'scrapy.squeues.PickleFifoDiskQueue', 'SCHEDULER_MEMORY_QUEUE': 'scrapy.squeues.FifoMemoryQueue', 'USER_AGENT': 'Crowl (+https://www.crowl.tech/)'}
Traceback (most recent call last):
 File "crowl.py", line 118, in <module>
 process.crawl(Crowler, **conf)
 File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 170, in crawl
 crawler = self.create_crawler(crawler_or_spidercls)
 File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 198, in create_crawler
 return self._create_crawler(crawler_or_spidercls)
 File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 203, in _create_crawler
 return Crawler(spidercls, self.settings)
 File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 55, in __init__
 self.extensions = ExtensionManager.from_crawler(self)
 File "/usr/lib/python3.7/site-packages/scrapy/middleware.py", line 58, in from_crawler
 return cls.from_settings(crawler.settings, crawler)
 File "/usr/lib/python3.7/site-packages/scrapy/middleware.py", line 34, in from_settings
 mwcls = load_object(clspath)
 File "/usr/lib/python3.7/site-packages/scrapy/utils/misc.py", line 44, in load_object
 mod = import_module(module)
 File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
 return _bootstrap._gcd_import(name[level:], package, level)
 File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
 File "<frozen importlib._bootstrap>", line 983, in _find_and_load
 File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 728, in exec_module
 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
 File "/usr/lib/python3.7/site-packages/scrapy/extensions/telnet.py", line 12, in <module>
 from twisted.conch import manhole, telnet
 File "/usr/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
 def write(self, data, async=False):
 ^
SyntaxError: invalid syntax

    Une idée pour régler ça?
    (Je suis sous manjaro)