SourceCode.support.convert_masterfiles_to_csv

Created on Wed Jan 16 17:10:10 2019

This script extracts all the data from FTT excel sheets in the “/In/FTTAssumptions/[model]” folders and saves them in separate csv files.

The user can select one or more scenarios to convert from the excel sheet:

1. If running this code as a script, set the input files in the final lines of the code

2. If the script is run at the start of the model run, it will automatically detect which files to convert, based on available csv files and whether the masterfile is newer than the csv files.

@author: MM and Femke Nijsse

Functions

are_csvs_older_than_masterfiles(...)	Find which files were last modified.
convert_1D_var_to_timeline(data, var, ...)	Some variables (e.g. TEWW, MEWW), are 1D in the excel sheets.
convert_masterfiles_to_csv(models[, ...])	The main function to convert masterfiles to csv files. Depending on how you run it, it can have three types of behaviour: a) If you run it as a script, it will overwrite files. It will ask for confirmation before overwriting the gamma values b) If you call it from another function b1) It will generate csvs if files don't exist yet b2) It will overwrite files if the masterfiles are newer than the csv files This will include gamma values.
costs_to_gam(data, var, reg, timeline_dict, ...)	In P, Tr, H and Fr, gamma values are not saved separately, but instead part of the cost variable.
csv_exists(file_path)	Check if a CSV file already exists.
directories_setup()	Set up directory masterfile and the general input directory
extract_data(raw_data, sheet_name, ...)	Extract a slice of data and convert it to a numpy array.
gamma_input_on_overwrite(out_dir, var, ...)	When the script is run as a script, and the gamma csv files already exist, confirm with the user if you want to overwrite, given that the user may not want to lose their calibrated gamma values
get_model_classification(variables_df)	Get the dimensions for the classifications
get_remaining_variables(vars_to_convert, ...)	Remove variables from the to-convert list if:
get_sheets_to_convert(var_dict, model, scen)	Get all the variables to convert to CSV files There are two options: 1. Convert all variables from excel sheet (for S0) 2. Convert only policy variables (for other scenarios). You can change which variables are in all scenarios in the FTT_variables file, column Scenario.
read_data(models, model, dir_masterfiles, ...)	Read the masterfiles
set_up_cols(model, var, var_dict, dims, ...)	Set up the size and name of a column.
set_up_rows(model, var, var_dict, dims)	Setting up the size of the rows, and the name of the rows
variable_setup(dir_masterfiles, models)	Set up the various containers and metadata for variables: variables_df_dict, var_dict, vars_to_convert, scenarios, timeline_dict
write_to_csv(data, row_title, col_title, ...)	Write the variables to a csv file, or to multiple csv files for 3D vars