The Truth about working as a Data Scientist
Let me start off by saying that I truly love the work that I am doing as a Data Scientist! I get to work on interesting technical problems that can highly impact people and businesses.
However, it is not all it's cracked up to be. There are quite a few people who have been transitioning to Data Science after it was called the sexiest job of the 21st century, only to become disillusioned with the field afterward!
In this article, I would like to guide you through the pros and cons of working as a Data Scientist. Hopefully, this will help you in getting a good view of what the field is about.
Do note that any cons or prons mentioned here might be different for you. This merely serves as a list of common things I have heard people say about this industry.
I am a pessimist, so let’s start with the cons 😅
Cons
Some of the cons might not be as bad as I make them sound. For example, I actually enjoy cleaning the data as it provides me with a good understanding of what I am working with.
However, to prevent any disillusioning, I think it is important that aspiring Data Scientists know the things that for many are clear disadvantages to working in this field.
1. Getting usable data
In practice, Data Scientists often take on the role of a data engineer. One important aspect of doing so is gathering the data necessary for performing your analyses/models.
This process can be quite tiresome as it requires either setting up large pipelines or lobbying business for this data.
Lobbying for data can be part of the job
Setting up these pipelines, although an interesting technical challenge, is typically not the challenge Data Scientists look for. Moreso, depending on the organization, this can be a big part of your day-to-day work!
2. Data is more often than not extremely messy
Data cleaning is a hugely underestimated part of Data Science. You can have all the state-of-the-art models at your disposal but it will always be a simple case of garbage-in = garbage-out.
Many professionals report that at least 80% of your work will be data cleaning. This might throw off aspiring Data Scientists who might think most of their work will include performing interesting analyses.
Datasets in industry are often too dirty for machine learning
Messy data is not always fixable. I have encountered many datasets in businesses where any form of machine learning simply is not useful.
When I was asked what insights I could gather from that data, my response was either “Get more data” or “Get better data”.
3. Involvement of Business
One largely misregarded component of working as a Data Scientist is the involvement of business. A big part of your work will most likely be dealing with the business. This can range from stakeholder and client management to lobbying for data.
The difficulty with this involvement is that expectations are generally quite high for you to solve large issues in the industry magically with data science.
Expect to spend significant time dashboarding
Although businesses might want to label everything as AI, in practice the actual use-cases for machine learning are few. This often means that you will be working on business analytics and dashboarding rather than data science.
4. Gatekeeping
Getting a job in the field of AI has become increasingly difficult over the last few years. 10 years ago you could get away with a few courses in data analytics whereas now you are expected to have a Ph.D., 10 years of experience in a FAANG, an active Github portfolio, and much more!
Recent graduates have difficulty finding work as a data scientist
Getting a job as a data scientist, or an analyst for that matter is difficult for an aspiring data scientist. The huge influx of recently graduated data scientists does not help this situation.
5. Lifelong learning
Lifelong learning sounds interesting but can be a cause of burnout-related issues. With the rapidly moving field of AI, you might be expected to be up-to-date on the latest research and constantly develop yourself in your spare time.
Being up-to-date in the field is near impossible
This constant need for development can get in the way of a good work-life balance. This is where many develop a feeling of missing out and are subject to the imposter syndrome.
Pros
Despite all these cons, there is still a reason data science and AI have been hyped; It is an amazingly interesting field that has the potential to revolutionize many industries!
So, let us go through a few pros you might not see in every list:
1. It is still a new field
Although data science has been hyped for quite a few years now, it is still not mature in many organizations. This phenomenon allows data scientists to be the first in their field, whether it is setting up a team or solving novel use cases.
Many opportunities for solving novel problems
These opportunities allow you to become an indispensable employee. Similarly, it gives you room for developing proof of concepts and trying out new techniques.
2. Impact
The impact of data science can be quite large. Especially if it is being used properly it can become the cornerstone of an organization. Even seemingly basic things like dashboarding can change the way management and product owners view their products.
Impact on business can be large
Although this is not easily achieved if spent enough time researching for which use-cases data science is the solution, a data scientist can be an important employee in the organization.
3. Opportunities
The role of a data scientist has changed over the last few years. Some work more as researchers whereas others focus more on data analysis. This discrepancy allows practitioners to explore many different roles. It is not uncommon for data scientists to also do the work of a data engineer or even a product owner.
There is significant flexibility in the role of a data scientist
If at some point, you want to focus more on business than as an individual contributor, it is very much possible to transition into a more business-oriented role as a data scientist.
Thank you for reading!
If you are, like me, passionate about AI, Data Science, or Psychology, please feel free to add me on LinkedIn or follow me on Twitter.