Python for Data Analysis Chapter 1: Preliminaries

Posted on by

What Is This Book About?

This book teaches you to become a data analyst using Python programming. To analyse data, we use Python to manipulate, process, clean and crunch data.

Data science is a term used to cover all statistical work and machine learning. There are many different methodologies which assist for each objective: this book is designed to prepare you with an overview of how to use Python for data analysis.

Data manipulation is also known as data analysis, wrangling, and munging.

What sort of data will we be working with? Python uses structured data, which means data that is organised in a certain way, for example: tables and spreadsheets, arrays and matrices, SQL databases, and time series data. Datasets can be organised into structured data for analysis, for example, we could have a newspaper archive which can be catalogued into a word frequency table.

Why Use Python for Data Analysis?

Python is an important language for data science, machine language, and software development. Although classed as a scripting language, like its fellow interpreted languages Perl and Ruby, it is a language that can be used to build serious software, using frameworks such as Django.

For statistical analysis, languages such as R, MATLAB, SAS, and Stata have always been popular, however since Python came on the scene in 1991, it has quickly caught up to be a serious contender for data analysis. Libraries such as pandas and scikit-learn mean Python can be used in place of these statistics-specific languages. Additionally, because Python also has the capability to build software, it can be used to build data applications without addons.

Python can be used as ‘glue’ to integrate other languages – for example, if we have legacy software written in FORTRAN, ‘glue code’ can be written to link the new code to the old to extend functionality.

Because R, MATLAB and SAS have long been used for data analysis, it has become common for code to be written in these languages, and then rewritten in Java or C#. Python has become a solution to the ‘two language problem’, where instead of using two different environments, one will suffice.

Python is not the best choice when speed and efficiency is crucial, as compiled languages like Java and C++ are much faster. However, programmer time is more valuable than CPU time, hence why Python is commonly used. If an application has low latency and/or has demanding resources, time spent coding a low-level language to achieve higher performance should be considered.

Building a concurrent, multithreaded application in Python can be difficult because of the GIL (Global Interpreter Lock). This prevents Python from executing more than one instruction at a time. Python can be used to execute multithreaded and parallel code with the help of Python C extensions.

This course can help you understand GIL: https://realpython.com/courses/understanding-global-interpreter-lock-gil/

Essential Python Libraries

  • NumPy (Numerical Python)
  • pandas
  • matplotlib
  • IPython and Jupyter
  • SciPy
  • scikit-learn
  • statsmodels

Installation and Setup

Community & Conferences

Navigating the Book