Utility functions#

Created on Wed Jul 19 04:43:43 2023

Copyright 2023 Roy Ruddle

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

vizdataquality.utils.detect_file_encoding(input_filename, read_in_chunks=True, confidence_level=0.9)#

Detect and return the encoding of a text file

Parameters:

  • input_filename (str) – The absolute path of the file.

  • read_in_chunks (boolean, optional) – True (read the whole file; more accurate) or False (read the file until the confidence level is satisfied). Default is False. The default is True.

  • confidence_level (float, optional) – Confidence threshold (only used if read_in_chunks = True). The default is 0.9.

Returns:

result – A dictionary containing the ‘encoding’ and a ‘confidence’ level, or None (file could not be found/opened).

Return type:

dict

vizdataquality.utils.end_logging(log, handlers)#

End logging.

Parameters:

  • log (Logger) – The logger.

  • handlers (TYPE) – The logfile handlers.

Return type:

None.

vizdataquality.utils.init_logging(logfile_name, overwrite_output_file=False)#

Initialise a logfile.

Logfiles can be useful when you are processing large datafiles or debugging.

Parameters:

  • logfile_name (str) – The full pathname of the logfile.

  • overwrite_output_file (boolean, optional) – True (start a new logfile) or False (append to the file if it exists, and start a new file if it does not exist). The default is False.

Returns:

  • log (Logger) – A Logger object.

  • handlers (list) – The logfile’s handlers.