Modified on
20 Jul 2022 06:48 pm
Skill-Lync
Built on top of the Python programming language, pandas is an open-source data analysis and manipulation tool that is quick, strong, flexible, and simple to use. The Pandas library helps perform mathematical operations in a flexible way. This open-source toolkit is utilised by data scientists to manipulate and analyse data in order to extract information from the data. It has a BSD licence and makes manipulating the number tables, simple. It was created using Python, Cython, and C. When compared to other libraries, it performs well and is open-source software. Instead of arrays, the library offers tables for memory retrieval thereby making the use of data structures easy.
The Pandas cheat sheet will walk you through the fundamentals of the Pandas library, from data structures to I/O, selection, removing indices or columns, sorting, and ranking, to applying functions and data alignment. It will also show you how to retrieve basic data about the data structures you're working with.
**dataset refers to the variable name of the data frame**
**PD alias pandas**
Use these commands to import data from a variety of different sources and formats.
dataset=pd.read_csv(filename) | From a CSV file
dataset=pd.read_table(filename) | From a delimited text file (like TSV)
dataset=pd.read_excel(filename) | From an Excel file
dataset=pd.read_sql(query, connection_object) | Read from a SQL table/database
dataset=pd.read_json(json_string) | Read from a JSON formatted string, URL or file.
dataset=pd.read_html(url) | Parses an html URL, string or file and extracts tables to a list of dataframes
dataset=pd.read_clipboard() | Takes the contents of your clipboard and passes it to read_table()
dataset=pd.DataFrame(dict) | From a dict, keys for columns names, values for data as lists
Use these commands to export a DataFrame to CSV, .xlsx, SQL, or JSON.
dataset.to_csv(filename) | Write to a CSV file
dataset.to_excel(filename) | Write to an Excel file
dataset.to_sql(table_name, connection_object) | Write to a SQL table
dataset.to_json(filename) | Write to a file in JSON format
These commands can be useful for creating test segments.
pd.DataFrame(np.random.rand(20,5)) | 5 columns and 20 rows of random floats
pd.Series(my_list) | Create a series from an iterable my_list
dataset.index = pd.date_range('1900/1/30', periods=dataset.shape[0]) | Add a date index
Use these commands to take a look at specific sections of your pandas DataFrame or Series.
dataset.head(n) | First n rows of the DataFrame
dataset.tail(n) | Last n rows of the DataFrame
dataset.shape | Number of rows and columns
dataset.info() | Index, Datatype and Memory information
dataset.describe() | Summary statistics for numerical columns
dataset.value_counts(dropna=False) | View unique values and counts
dataset.apply(pd.Series.value_counts) | Unique values and counts for all columns
dataset[col] | Returns column with label col as Series
dataset[[col1, col2]] | Returns columns as a new DataFrame
dataset.iloc[0] | Selection by position
dataset.loc['index_one'] | Selection by index
dataset.iloc[0,:] | First row
dataset.iloc[0,0] | First element of first column
dataset.columns = ['a','b','c'] | Rename columns
pd.isnull() | Checks for null Values, Returns Boolean Arrray
pd.notnull() | Opposite of pd.isnull()
dataset.dropna() | Drop all rows that contain null values
dataset.dropna(axis=1) | Drop all columns that contain null values
dataset.dropna(axis=1,thresh=n) | Drop all rows have have less than n non null values
dataset.fillna(x) | Replace all null values with x
dataset.fillna(s.mean()) | Replace all null values with the mean
dataset.astype(float) | Convert the datatype of the series to float
dataset.replace(1,'one') | Replace all values equal to 1 with ‘one’
dataset.replace([1,3],['one','three']) | Replace all 1 with ‘one’ and 3 with ‘three’
dataset.rename(columns=lambda x: x + 1) | Mass renaming of columns
dataset.rename(columns={'old_name': 'new_ name'}) | Selective renaming
dataset.set_index('column_one') | Change the index
dataset.rename(index=lambda x: x + 1) | Mass renaming of index
dataset[dataset[col] > x] | Rows where the column col is greater than x
dataset[(dataset[col] > x) & (dataset[col] < y)] | Rows where y > col > x
dataset.sort_values(col1) | Sort values by col1 in ascending order
dataset.sort_values(col2,ascending=False) | Sort values by col2 in descending order
dataset.sort_values([col1,col2],ascending=[True,False]) | Sort values by col1 in ascending order then col2 in descending order
dataset.groupby(col) | Returns a groupby object for values from one column
dataset.groupby([col1,col2]) | Returns groupby object for values from multiple columns
dataset.groupby(col1)[col2] | Returns the mean of the values in col2, grouped by the values in col1 (mean can be replaced with almost any function from the statistics module)
dataset.pivot_table(index=col1,values=[col2,col3],aggfunc=mean) | Create a pivot table that groups by col1 and calculates the mean of col2 and col3
dataset.groupby(col1).agg(np.mean) | Find the average across all columns for every unique col1 group
dataset.apply(np.mean) | Apply np.mean() across each column
dataset.apply(np.max,axis=1) | Apply np.max() across each row
Combining multiple datasets into 1 single dataset
dataset1.append(dataset2) | Add the rows in dataset1 to the end of dataset2 (columns should be identical)
pd.concat([dataset1, dataset2],axis=1) | Add the columns in dataset1 to the end of dataset2 (rows should be identical)
dataset1.join(dataset2,on=col1,how='inner') | SQL-style join the columns in dataset1 with the columns on dataset2 where the rows for col have identical values. 'how' can be one of 'left', 'right', 'outer', 'inner'
dataset.describe() | Summary statistics for numerical columns
dataset.mean() | Returns the mean of all columns
dataset.corr() | Returns the correlation between columns in a DataFrame
dataset.count() | Returns the number of non-null values in each DataFrame column
dataset.max() | Returns the highest value in each column
dataset.min() | Returns the lowest value in each column
dataset.median() | Returns the median of each column
dataset.std() | Returns the standard deviation of each column
Author
Navin Baskar
Author
Skill-Lync
Subscribe to Our Free Newsletter
Continue Reading
Related Blogs
Premium Master’s Program can do so at a discount of 20%. But, Christmas is time for sharing, therefore if you and your friend were to join any Skill-Lync Master’s Program together, both of you will get a discount of 30% on the course fee of your Premium Master’s Program
24 Dec 2021
Increase your career opportunities by becoming a software engineer and make the world a better place. Enroll in upskilling courses and practice the skills you learn.
27 Dec 2021
Software development is rated as the best job in the industry. Individuals with the right software development skills, good communication, and an open mind to adapt, learn, and evolve can find success in the field.
28 Dec 2021
If you aspire for a career in the software development space, upskilling yourself with the knowledge and practical application of programming languages is mandatory.
29 Dec 2021
The most fascinating thing about the chosen ways of completing tasks on computers is that we only choose them because we do not have a simpler way yet.
30 Dec 2021
Author
Skill-Lync
Subscribe to Our Free Newsletter
Continue Reading
Related Blogs
Premium Master’s Program can do so at a discount of 20%. But, Christmas is time for sharing, therefore if you and your friend were to join any Skill-Lync Master’s Program together, both of you will get a discount of 30% on the course fee of your Premium Master’s Program
24 Dec 2021
Increase your career opportunities by becoming a software engineer and make the world a better place. Enroll in upskilling courses and practice the skills you learn.
27 Dec 2021
Software development is rated as the best job in the industry. Individuals with the right software development skills, good communication, and an open mind to adapt, learn, and evolve can find success in the field.
28 Dec 2021
If you aspire for a career in the software development space, upskilling yourself with the knowledge and practical application of programming languages is mandatory.
29 Dec 2021
The most fascinating thing about the chosen ways of completing tasks on computers is that we only choose them because we do not have a simpler way yet.
30 Dec 2021
Related Courses