Skip to content

[WIP] feat: PII detection and de-identification (Presidio) — #233 - #248

Draft
NKeleher wants to merge 7 commits into
feat/python-replicationfrom
feat/pii-deidentification
Draft

[WIP] feat: PII detection and de-identification (Presidio) — #233#248
NKeleher wants to merge 7 commits into
feat/python-replicationfrom
feat/pii-deidentification

fix: apply hash/code PII decisions as prep steps (visible in change log)

f49916d
Select commit
Loading
Failed to load commit list.
SonarQubeCloud / SonarCloud Code Analysis failed Jul 18, 2026 in 39s

Quality Gate failed

Failed conditions
1 New Vulnerabilities (required ≤ 0)
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Annotations

Check warning on line 24 in src/datasure/replication/readme.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Function "generate_readme" has 16 parameters, which is greater than the 13 authorized. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVjnI9ruSZJpsfHi&open=AZ9yYVjnI9ruSZJpsfHi&pullRequest=248

Check failure on line 152 in src/datasure/replication/py_script_generators.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal "from pathlib import Path" 3 times. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVmhI9ruSZJpsfHk&open=AZ9yYVmhI9ruSZJpsfHk&pullRequest=248

Check failure on line 378 in src/datasure/processing/pii.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 22 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVnVI9ruSZJpsfHn&open=AZ9yYVnVI9ruSZJpsfHn&pullRequest=248

Check failure on line 424 in src/datasure/replication/py_prep_script_generator.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 19 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVmoI9ruSZJpsfHl&open=AZ9yYVmoI9ruSZJpsfHl&pullRequest=248

Check failure on line 726 in src/datasure/processing/pii.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 19 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMqz6UF9MhVQckIH&open=AZ9yfMqz6UF9MhVQckIH&pullRequest=248

Check failure on line 344 in src/datasure/processing/pii.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Reraise this exception to stop the application as the user expects 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVnVI9ruSZJpsfHm&open=AZ9yYVnVI9ruSZJpsfHm&pullRequest=248

Check warning on line 710 in src/datasure/processing/pii.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Make sure that using this pseudorandom number generator is safe here. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMqz6UF9MhVQckIG&open=AZ9yfMqz6UF9MhVQckIG&pullRequest=248

Check failure on line 796 in src/datasure/processing/pii.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 18 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMqz6UF9MhVQckII&open=AZ9yfMqz6UF9MhVQckII&pullRequest=248

Check failure on line 310 in src/datasure/replication/py_script_generators.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal " )" 4 times. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMp76UF9MhVQckIE&open=AZ9yfMp76UF9MhVQckIE&pullRequest=248

Check failure on line 334 in src/datasure/replication/py_script_generators.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal " .alias(column)" 3 times. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMp76UF9MhVQckIF&open=AZ9yfMp76UF9MhVQckIF&pullRequest=248

Check failure on line 203 in src/datasure/replication/package_builder.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 31 to the 15 allowed. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yYVmWI9ruSZJpsfHj&open=AZ9yYVmWI9ruSZJpsfHj&pullRequest=248

Check failure on line 327 in src/datasure/replication/py_script_generators.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal " df = df.with_columns(" 3 times. 

See more on https://sonarcloud.io/project/issues?id=PovertyAction_datasure&issues=AZ9yfMp76UF9MhVQckID&open=AZ9yfMp76UF9MhVQckID&pullRequest=248
View more details on SonarQubeCloud