To what extent can I learn Python to work?

1. Establish goals and understand needs

Whenever you do anything, you must first determine the goals so as not to lose your way. Our job target is Python crawler engineer.

Find relevant job requirements on some large domestic recruitment websites:

Looking carefully, we can draw the following points:

1. Python is not The only thing that can be used for crawling is many languages, especially Java. Mastering them at the same time and having relevant development experience are very important bonuses

2. Most companies require a certain level of crawler technology. Depth and breadth, depth refers to technologies such as anti-crawling, encryption cracking, login verification, etc.; breadth refers to distributed, cloud computing, etc., these are all bonus points

3. Crawlers, not crawling Once you have the data, you are done. If you have experience in data extraction, cleaning, deduplication, etc., it will also be a bonus.

4. Generally, companies will have their own crawler systems, and new employees need to learn from them. A common job is to maintain the crawler system, which you must understand

5. The last bonus item is front-end knowledge, especially commonly used js, ajax, html/xhtml, css and other related technologies. Good, among which familiarity with js code is very important

6. Add one more thing, as the market share of handheld devices becomes higher and higher, the proficiency in using app data collection and packet capture tools will become more and more important. The more important it is

It is not required to master all the above content, but the more you master, the higher your importance

2. Regarding the four points of Python interview, you can do it Everything will be fine!

First point: Python

Because the interview is for a Python crawler position, most interviewers will test the interviewer’s basic Python knowledge, including but not limited to:

The difference between Python2. and the like

Python threads

The second point: data structures and algorithms

Data structures and algorithms are an important question for interviewers, especially school admissions interviews. A very important point. Of course, small companies will not care too much about these. Judging from the current recruitment situation, the importance attached to the interviewer's data structure and algorithm is directly proportional to the quality of the company. Those who never ask you about the data structure will be ignored. Be careful if they treat you as a coder. Of course, the above situation is not absolute, and the final interpretation rights belong to the interviewer.

The third point: Python crawler

The most important and critical point is of course your Python crawler-related knowledge and experience reserves. This is usually the focus of the interviewer's investigation, including But not limited to:

What anti-crawler strategies have you encountered?

What are your commonly used anti-anti-crawler solutions?

Do you use multi-threading and asynchronous? In addition, what other methods have you used to improve crawler efficiency?

Have you ever done incremental crawling?

Do you know the Python crawler framework?

The fourth point: crawler-related project experience

Crawler focuses on practice. In addition to theoretical knowledge, the interviewer will also pay great attention to crawler-related projects:

< p>What crawler projects have you done? It would be best if there was Github

Which do you think is the best crawler project you have ever done? What problems were solved? What's so special about it?