Script_Register.csv

Script Name,Script Location Path,Input File(s),Output File(s),Comment
create_clean_chartevents.ipynb,mimic_data_preprocessing/create_clean_chartevents.ipynb,CHARTEVENTS.csv,"01) chartevents_subset.parquet
02) chartevents_subset_measurement_count.parquet
03) chartevents_cleaning_02_multiple_measurements_only.parquet
04) flagged_values.parquet
05) chartevents_cleaning_03_values_in_valid_range.parquet
06) threshold_swap_candidates.parquet
07) threshold_swap_data.parquet
08) thresholds_fixed.parquet
09) chartevents_cleaning_04_exact_threshold_swaps_reverted.parquet
10) chartevents_cleaning_05_thresholds_in_valid_range.parquet
11) threshold_removal_candidates.parquet
12) threshold_removal_data.parquet
13) thresholds_to_be_removed.parquet
14) chartevents_cleaning_06_overlapping_thresholds_removed.parquet
15) chartevents_clean.parquet",Prepare the MIMIC-III data set CHARTEVENTS.csv for later use in the master project context by cleaning the vital parameter values and alarm threshold values it contains
add_dif_between_measurements.ipynb,mimic_data_preprocessing/add_dif_between_measurements.ipynb,"01) chartevents_subset.parquet
02) unique_icustays_in_chartevents_subset.parquet",chunk_analysis_data.parquet,File that is used to derive possible chunking rules
create_chunks.ipynb,mimic_data_preprocessing/create_chunks.ipynb,"01) chartevents_clean.parquet
02) unique_icustays_in_chartevents_subset.parquet","01) chartevents_clean_values_with_chunkid_65.parquet
02) chartevents_clean_values_and_thresholds_with_chunkid_65.parquet",65 in Putput File refers to difference to next measurement after which is being chunked
resample_cleaned_chartevents_chunks.ipynb,mimic_data_preprocessing/resample_cleaned_chartevents_chunks.ipynb,chartevents_clean_values_and_thresholds_with_chunkid_65.parquet,"01) chartevents_clean_values_and_thresholds_with_chunkid_65_resampled_median.parquet
02) chartevents_clean_values_and_thresholds_with_chunkid_65_resampled_mean.parquet","Resample previously cleaned and chunked CHARTEVENTS data as they originally have an inconsistent sampling rate, but forecasting requires a consistent sampling rate. Output filename suffix refers to the method used in downsampling (median or mean)."
chartevents_analysis.ipynb,mimic_chartevents_analysis/chartevents_analysis.ipynb,"01) chartevents_subset.parquet
02) unique_icustays_in_chartevents_subset.parquet",,Descriptive statistics for chartevents_subset.parquet (general and for three relevant parameters)
compute_unique_icustays_in_chartevents_subset.ipynb,mimic_chartevents_analysis/compute_unique_icustays_in_chartevents_subset.ipynb,chartevents_subset.parquet,unique_icustays_in_chartevents_subset.parquet,Compute unqiue ICU stays in chartevents_subset and save it for quick reuse in other scripts without having to run the calculation again.
create_sampling_rates.ipynb,mimic_chartevents_analysis/create_sampling_rates.ipynb,"01) chartevents_subset.parquet
02) unique_icustays_in_chartevents_subset.parquet",sampling_rates_for_icustay_itemid.parquet,
create_sampling_rates_for_chunks.ipynb,mimic_chartevents_analysis/create_sampling_rates_for_chunks.ipynb,chartevent_subset_values_with_chunkid_65.parquet,sampling_rates_for_chunkid.parquet,Same file as create_sampling_rates.ipynb just adapted to chunks
analyze_sampling_rates.ipynb,mimic_chartevents_analysis/analyze_sampling_rates.ipynb,sampling_rates_for_icustay_itemid.parquet,none,visualize boxplot and descriptive statistics
analyze_sampling_rates_for_chunks.ipynb,mimic_chartevents_analysis/analyze_sampling_rates_for_chunks.ipynb,sampling_rates_for_chunkid.parquet,none,Same file as analyze_sampling_rate just adapted to chunks
analyze_chunks_by_icustay.ipynb,mimic_chartevents_analysis/analyze_chunks_by_icustay.ipynb,chartevent_subset_values_with_chunkid_65.parquet,none,
derive_chunking_rules.ipynb,mimic_chartevents_analysis/derive_chunking_rules.ipynb,chunk_analysis_data.parquet,,proposes two chunking options (after 60 mins and after 120 mins)
plots_for_resampled_cleaned_chartevents_chunks.ipynb,mimic_chartevents_analysis/plots_for_resampled_cleaned_chartevents_chunks.ipynb,"01) chartevents_clean_values_and_thresholds_with_chunkid_65.parquet
02) chartevents_clean_values_and_thresholds_with_chunkid_65_resampled_median.parquet
03) chartevents_clean_values_and_thresholds_with_chunkid_65_resampled_mean.parquet",none,"Create time series plots, which allow to compare vital parameter values series before and after resampling"
generate_alarm_data.py,mimic_alarm_data_generation/generate_alarm_data.py,"01) chartevents_clean.parquet
02) chartevents_clean_values_and_thresholds_with_chunkid_65.parquet
03) unique_icustays_in_chartevents_subset.parquet","01) alarm_data.csv,
02) alarm_data_with_chunks_65.csv",Execute with chunked version of cleaned chartevents via parameter --chunks
verify_alarm_data.ipynb,mimic_alarm_data_generation/verify_alarm_data.ipynb,"01) alarm_violations.csv
02) alarm_data.csv
03) alarm_data_with_chunks_65.csv
04) unique_icustays_in_chartevents_subset.parquet
05) chartevents_clean.parquet",,Only internal verification
01_prescriptions_analysis.ipynb,mimic_medication_analysis/prescriptions_analysis.ipynb,"01) PRESCRIPTIONS.csv
02) unique_icustays_in_chartevents_subset.parquet
03) prescriptions_based_medications.parquet
04) chartevents_subset.parquet",prescriptions_based_medications.parquet,Analyze PRESCRIPTIONS.csv for introduction of medication flag
02_inputevents_mv_analysis.ipynb,mimic_medication_analysis/inputevents_mv_analysis.ipynb,"01) D_ITEMS.csv
02) INPUTEVENTS_MV.csv
03) unique_icustays_in_chartevents_subset.parquet
04) CHARTEVENTS.csv
05) inputevents_based_medications.parquet
06) chartevents_subset.parquet",inputevents_based_medications.parquet,"Analyze INPUTEVENTS_MV.csv for introduction of medication flag: Based on this analysis and its produced output, we added flag indicating medication within time period of 1 hour before and after an alarm"
03_integrate_med_flag.ipynb,mimic_medication_analysis/integrate_med_flag.ipynb,"01) alarm_data.csv
02) inputevents_based_medications.parquet",alarm_data_with_med_{time window}min.parquet,Time window of currently 120 minutes should be adjusted if we want to use this flag later
alarm_violations_analysis.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis.ipynb,alarm_violations.csv,,
alarm_violations_analysis_HR.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis_HR.ipynb,alarm_violations.csv,,
alarm_violations_analysis_MV.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis_MV.ipynb,alarm_violations.csv,,Did not investigated thresholds as we decided to not include this parameter into further analyses
alarm_violations_analysis_NBPs.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis_NBPs.ipynb,alarm_violations.csv,,
alarm_violations_analysis_O2SAT.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis_O2SAT.ipynb,alarm_violations.csv,,
alarm_violations_analysis_RR.ipynb,mimic_alarm_violations_analysis/alarm_violations_analysis_RR.ipynb,alarm_violations.csv,,
prediction_rnnmodel_normal.py,mimic_prediction_rnn/prediction_normal.py,"01) resample_output_{parameter}_first{n_chunks}.parquet
02) 04_pre-trained_model_normal_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_normal_window{window_idx}.pickle","01) 01_train_series_normal_window{window_idx}.pickle
02) 02_pred_series_normal_window{window_idx}.pickle
03) 04_pre-trained_model_normal_window{window_idx}.pickle
04) 05_prediction_{chunk_id}_normal_window{window_idx}.pickle
05) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_normal_window{window_idx}.pickle
06) confusion_matrix_models_normal_{endogenous_input}.pickle",RNNModel prediction with median resampled non-scaled chunks as endogenous variables
prediction_rnnmodel_normal_covariates.py,mimic_prediction_rnn/prediction_normal_covariates.py,"01) resample_output_{parameter}_first{n_chunks}.parquet
02) 04_pre-trained_model_{alarm_type}_normal_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_normal_window{window_idx}.pickle","01) 01_train_series_{endo/exo}[_{alarm_type}]_normal_window{window_idx}.pickle
02) 02_pred_series_{endo/exo}[_{alarm_type}]_normal_window{window_idx}.pickle
03) 04_pre-trained_model_{alarm_type}_normal_window{window_idx}.pickle
04) 05_prediction_{chunk_id}_{alarm_type}_normal_window{window_idx}.pickle
05) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_normal_window{window_idx}.pickle
06) confusion_matrix_models_normal_{endogenous_input}.pickle",RNNModel prediction with max and min resampled non-scaled chunks as endogenous variables and median resampled non-scaled chunks as exogenous variables
prediction_rnnmodel_scaled_s1.py,mimic_prediction_rnn/prediction_scaled_s1.py,"01) means_z_scaling-pickle
02) stds_z_scaling.pickle
03) {parameter}_{window_idx}_{train/pred}_median.pickle
04) 04_pre-trained_model_scaled_s1_window{window_idx}.pickle
05) resample_output_{parameter}_first{n_chunks}.parquet
06) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s1.pickle","01) 04_pre-trained_model_scaled_s1_window{window_idx}.pickle
02) 05_prediction_{chunk_id}_scaled_s1_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s1.pickle
04) confusion_matrix_models_scaled_{endogenous_input}_s1.pickle",RNNModel prediction with median resampled z-scaled chunks as endogenous variables
prediction_rnnmodel_scaled_covariates_s1.py,mimic_prediction_rnn/prediction_scaled_covariates_s1.py,"01) means_z_scaling-pickle
02) stds_z_scaling.pickle
03) {parameter}_{window_idx}_{train/pred}_{median/min/max}.pickle
04) 04_pre-trained_model_{high/low}_scaled_s1_window{window_idx}.pickle
05) resample_output_{parameter}_first{n_chunks}.parquet
06) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s1.pickle","01) 04_pre-trained_model_{high/low}_scaled_s1_window{window_idx}.pickle
02) 05_prediction_{chunk_id}_{high/low}_scaled_s1_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s1.pickle
04) confusion_matrix_models_scaled_{endogenous_input}_s1.pickle",RNNModel prediction with max and min resampled z-scaled chunks as endogenous variables and median resampled scaled chunks as exogenous variables
prediction_rnnmodel_scaled_s2.py,mimic_prediction_rnn/prediction_scaled_s2.py,"01) resample_output_{parameter}_first{n_chunks}.parquet
02) 04_pre-trained_model_scaled_s2_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s2.pickle","01) 01_train_series_scaled_window{window_idx}.pickle
02) 02_pred_series_scaled_window{window_idx}.pickle
03) 03_pred_scaler_window{window_idx}.pickle
04) 04_pre-trained_model_scaled_s2_window{window_idx}.pickle
05) 05_prediction_{chunk_id}_scaled_s2_window{window_idx}.pickle
06) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s2.pickle
07) confusion_matrix_models_scaled_{endogenous_input}_s2.pickle",RNNModel prediction with median resampled min-max-scaled chunks as endogenous variables
prediction_rnnmodel_scaled_covariates_s2.py,mimic_prediction_rnn/prediction_scaled_covariates_s2.py,"01) resample_output_{parameter}_first{n_chunks}.parquet
02) 04_pre-trained_model_{alarm_type}_scaled_s2_window{window_idx}.pickle
03) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s2.pickle","01) 01_train_series_{endo/exo}[_{alarm_type}]_scaled_window{window_idx}.pickle
02) 02_pred_series_{endo/exo}[_{alarm_type}]_scaled_window{window_idx}.pickle
03) 03_pred_scaler_window{window_idx}.pickle
04) 04_pre-trained_model_{alarm_type}_scaled_s2_window{window_idx}.pickle
05) 05_prediction_{chunk_id}_{alarm_type}_scaled_s2_window{window_idx}.pickle
06) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_scaled_window{window_idx}_s2.pickle
07) confusion_matrix_models_scaled_{endogenous_input}_s2.pickle",RNNModel prediction with max and min resampled min-maxscaled chunks as endogenous variables and median resampled scaled chunks as exogenous variables
analyze_model_level_predictions.iypnb,mimic_prediction_rnn/analyze_model_level_predictions.iypnb,confusion_matrix_models_{version}_{endogenous_input}_s2.pickle,rnn_model_results_{n_chunks}_{style}.parquet,Analysis and visualization of model-level results
analyze_chunk_level_predictions.iypnb,mimic_prediction_rnn/analyze_chunk_level_predictions.iypnb,"01) confusion_matrix_chunks_{model_type}_{parameter}_{endogenous_input}_{version}_window{window_idx}_s2.pickle
02) resample_output_{parameter}_first{n_chunks}.parquet",,Analysis and visualization of chunk-level results