Blog Logo
TAGS

Quinn - Pyspark helper methods to maximize developer productivity

Quinn provides DataFrame validation functions, useful column functions / DataFrame transformations, and performant helper functions to maximize developer productivity. It includes functions like validate_presence_of_columns(), validate_schema(), validate_absence_of_columns(), single_space(), remove_all_whitespace(), anti_trim(), remove_non_word_characters(), multi_equals(), and approx_equal(). It also has an array_choice() function that returns a PySpark column with a random value from the input column parameter. Quinn is available on PyPI and can be installed using pip install quinn.