Skip to content

Add proxy support to search and scraper#104

Open
ramicaza wants to merge 7 commits into
trevorhobenshield:mainfrom
ramicaza:main
Open

Add proxy support to search and scraper#104
ramicaza wants to merge 7 commits into
trevorhobenshield:mainfrom
ramicaza:main

Conversation

@ramicaza

@ramicaza ramicaza commented Jul 4, 2023

Copy link
Copy Markdown

The hacky patch I mentioned in #97 couldn't get me all the way so I decided to fork. Hope I didn't miss anything while adding the proxy.

@italianconcerto

Copy link
Copy Markdown

Hey, is it possible to merge this branch?

@italianconcerto

Copy link
Copy Markdown

@trevorhobenshield

Comment thread twitter/scraper.py
ids = set(find_key(initial_data, 'rest_id'))
cursor = get_cursor(initial_data)
except Exception as e:
self.logger.error('Failed to get initial pagination data', e)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this intentional?

@sdfgsdfsdf

Copy link
Copy Markdown

is it possible to merge this branch?

@trevorhobenshield
trevorhobenshield marked this pull request as draft February 10, 2024 15:06
@trevorhobenshield
trevorhobenshield marked this pull request as ready for review February 10, 2024 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants