Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions analysis/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/)

## [unreleased] (Added 🚀 | Changed | Removed | Fixed 🐞 | Chore 👨‍💻 👩‍💻)

### Added 🚀

- **Who made each commit**: commit changes also hold each commit's type, hotfix flag, author and co-authors, under the
same invented names as in author ownership, or the real names with `--with-author-names`.

### Fixed 🐞

- **Kotlin classes after annotations with arguments**: `dependencyparser` reads a class written after several
annotations, one of them with arguments such as `@Timeout(120)`, as the class it is instead of losing it.

## [2.4.0] - 2026-11-10

### Added 🚀
Expand Down
22 changes: 14 additions & 8 deletions analysis/analysers/parsers/GitLogParser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The names are pseudonymous, not anonymous: anyone with the repository can match
still be personal data under GDPR. Treat and share it accordingly.

`--with-author-names`, next to `--author-ownership`, opts out of this: the map then holds real names as git has them, under
a clustering titled **Author ownership (real names)**, and still no emails. The command warns about it last, even with
a clustering titled **Author ownership (real names)** and in the commit changes, and still no emails. The command warns about it last, even with
`--silent`, and the Explorer's Clusters tab shows a warning for such a map.


Expand All @@ -58,11 +58,17 @@ git 2.37 or newer.
empty file counts as wholly changed.
- Merge commits are left out, as their commits are listed themselves. Changes follow a renamed file to its path today,
and a pure rename changes nothing. A commit with no file left today is not listed.
- Each commit also carries its date and how many files and lines it changed in all, including files gone today.

Commit subjects enter the map, and anyone with the repository can match a hash to its author, so a map may still be
personal data under GDPR. The command says so last. On a shallow clone the oldest commit counts every file as changed,
and the command warns about it.
- Each commit also carries its date, its author, its co-authors and how many files and lines it changed in all,
including files gone today. The author is the one git records, and the co-authors are everyone else its
`Co-authored-by` trailers name, in their order, each once and joined by `; `, as in `Teal Falcon; Jade Owl`, both after
applying `.mailmap`. A commit without co-authors carries none. Everyone appears under the same invented name the
author ownership gives them, whether or not the map has author ownership. With `--with-author-names`, next to
`--author-ownership`, they appear under their real names.

The invented names are pseudonymous, not anonymous: anyone with the repository can match them and the hashes to people,
and commit subjects are free text, so a map may still be personal data under GDPR. The command says so last. On a
shallow clone the oldest commit counts every file as changed and the invented names are not stable, and the command
warns about it.
## Usage and Parameters

As this parser can generate metrics based on a given git-log file or given a git repository (where the git-log is then generated on the fly), some parameters are only available in the log-scan mode or the repo-scan mode.
Expand All @@ -73,8 +79,8 @@ As this parser can generate metrics based on a given git-log file or given a git
| `repo-scan` | use repo-scan mode - generates cc.json from an automatically generated git-log file |
| `--add-author` | add an array of authors to every file |
| `--author-ownership` | add each author's share of each file under an invented name, from `git blame` (repo-scan only) |
| `--with-author-names` | write real author names instead of numbers (needs `--author-ownership`) |
| `--commit-changes-since YYYY-MM-DD` | add what each commit since this day changed in each file (repo-scan only) |
| `--with-author-names` | write real author names instead of invented ones, also for commit changes (needs `--author-ownership`) |
| `--commit-changes-since YYYY-MM-DD` | add what each commit since this day changed in each file, and who made it (repo-scan only) |
| `--git-log=FILE` | git-log file to parse (only available for log-scan mode!) |
| `-h, --help` | displays help and exits |
| `-nc, --not-compressed` | save uncompressed output File |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import de.maibornwolff.codecharta.analysers.analyserinterface.AnalyserDialogInte
import de.maibornwolff.codecharta.analysers.analyserinterface.AnalyserInterface
import de.maibornwolff.codecharta.analysers.analyserinterface.commit.CommitAnalysisContext
import de.maibornwolff.codecharta.analysers.analyserinterface.commit.GitWorktreeManager
import de.maibornwolff.codecharta.analysers.parsers.gitlog.authors.AuthorHistory
import de.maibornwolff.codecharta.analysers.parsers.gitlog.commitchanges.CommitChangesScan
import de.maibornwolff.codecharta.analysers.parsers.gitlog.ownership.AuthorOwnershipScan
import de.maibornwolff.codecharta.analysers.parsers.gitlog.util.GitAdapter
Expand Down Expand Up @@ -61,7 +62,10 @@ class RepoScanCommand(private val printsClusterHints: Boolean = true) : Analyser

@CommandLine.Option(
names = [AuthorOwnershipScan.NAMES_FLAG],
description = ["write the real names of the authors into the map instead (personal data, needs --author-ownership)"]
description = [
"write the real names of the authors into the map instead, also for commit changes " +
"(personal data, needs ${AuthorOwnershipScan.FLAG})"
]
)
private var withAuthorNames = false

Expand Down Expand Up @@ -101,11 +105,12 @@ class RepoScanCommand(private val printsClusterHints: Boolean = true) : Analyser
if (!silent) println("Creating git-ls file...")
val gitLsFile = createGitLsFile(context.inputDir.toPath())

val ownershipScan = AuthorOwnershipScan(context.inputDir, withAuthorNames).takeIf { authorOwnership }
val authorHistory = AuthorHistory(context.inputDir)
val ownershipScan = AuthorOwnershipScan(context.inputDir, withAuthorNames, authorHistory).takeIf { authorOwnership }
if (!silent && ownershipScan != null) println("Blaming files...")
val ownership = ownershipScan?.lens() ?: ClustersLens()

val commitChangesScan = commitChangesSince?.let { CommitChangesScan(context.inputDir, it) }
val commitChangesScan = commitChangesSince?.let { CommitChangesScan(context.inputDir, it, withAuthorNames, authorHistory) }
if (!silent && commitChangesScan != null) println("Reading the changes of each commit...")
val commitChanges = commitChangesScan?.lens() ?: ClustersLens()

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
package de.maibornwolff.codecharta.analysers.parsers.gitlog.authors

import de.maibornwolff.codecharta.analysers.parsers.gitlog.util.GitCommand
import java.io.File

/** A commit's author and co-authors, the author first. */
internal data class CommitPeople(val hash: String, val people: List<String>)

/** Reads who made each commit of the repository's history, with the mailmap applied, and reads it only once. */
class AuthorHistory(repositoryFolder: File) {
private val git = GitCommand(repositoryFolder)

internal val commitsOldestFirst: List<CommitPeople> by lazy { readCommitsOldestFirst() }

private fun readCommitsOldestFirst(): List<CommitPeople> {
val format = "--format=%H%x00%aN%x00%(trailers:key=Co-authored-by,valueonly,separator=%x01)"
val rows = git
.outputOrNull("-c", "i18n.logOutputEncoding=UTF-8", "log", "--reverse", "--topo-order", format) { output ->
output.bufferedReader(Charsets.UTF_8).useLines { lines -> lines.map { it.split(NUL) }.toList() }
}.orEmpty()
val nameByCoAuthor = mappedNames(rows.flatMap { coAuthorsIn(it) }.distinct())
return rows.map { row ->
CommitPeople(row[0], (listOf(row[1]) + coAuthorsIn(row).map { nameByCoAuthor.getValue(it) }).distinct())
}
}

private fun coAuthorsIn(logRow: List<String>): List<String> =
logRow[2].split(TRAILER_SEPARATOR).map { it.trim() }.filter { it.isNotEmpty() }

// Git applies the mailmap to a commit's author but not to its trailers, so the co-authors are mapped separately
private fun mappedNames(coAuthors: List<String>): Map<String, String> {
val withEmail = coAuthors.filter { CONTACT_WITH_EMAIL.matches(it) }
val input = File.createTempFile("co-authors", ".txt")
try {
input.writeText(withEmail.joinToString("\n", postfix = "\n"), Charsets.UTF_8)
val mapped = git.outputOrNull("check-mailmap", "--stdin", input = input) { it.bufferedReader(Charsets.UTF_8).readLines() }
val contacts = if (mapped?.size == withEmail.size) withEmail.zip(mapped).toMap() else emptyMap()
return coAuthors.associateWith { nameOf(contacts[it] ?: it) }
} finally {
if (!input.delete()) input.deleteOnExit()
}
}

private fun nameOf(contact: String): String = contact.substringBefore(EMAIL_START).trim()

companion object {
private const val NUL = "\u0000"
private const val TRAILER_SEPARATOR = "\u0001"
private const val EMAIL_START = "<"
private val CONTACT_WITH_EMAIL = Regex(".+<.+>")
}
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package de.maibornwolff.codecharta.analysers.parsers.gitlog.ownership
package de.maibornwolff.codecharta.analysers.parsers.gitlog.authors

import java.util.Random

Expand Down Expand Up @@ -26,6 +26,17 @@ internal class AuthorPseudonyms(firstCommitHash: String) {
private fun String.capitalised() = replaceFirstChar { it.uppercase() }

companion object {
/** The pseudonym of everyone in [commitsOldestFirst], by their position in the order of first commit. */
fun byPerson(commitsOldestFirst: List<CommitPeople>): Map<String, String> {
val firstCommitHash = commitsOldestFirst.firstOrNull()?.hash ?: return emptyMap()
val pseudonyms = AuthorPseudonyms(firstCommitHash)
return commitsOldestFirst
.flatMap { it.people }
.distinct()
.withIndex()
.associate { (position, person) -> person to pseudonyms.nameOf(position) }
}

val COLOURS = listOf(
"amber", "apricot", "aqua", "azure", "bronze", "burgundy", "cerise", "cerulean", "claret", "cobalt", "copper", "coral",
"crimson", "cyan", "denim", "emerald", "fern", "flax", "fuchsia", "garnet", "gold", "heather", "indigo", "iris", "jade",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package de.maibornwolff.codecharta.analysers.parsers.gitlog.commitchanges

import de.maibornwolff.codecharta.analysers.parsers.gitlog.input.metrics.DefaultSemanticCommitStyle
import de.maibornwolff.codecharta.analysers.parsers.gitlog.input.metrics.HotfixCommits
import de.maibornwolff.codecharta.model.AttributeDescriptor
import de.maibornwolff.codecharta.model.AttributeType
import de.maibornwolff.codecharta.model.Cluster
Expand All @@ -24,20 +26,37 @@ internal object CommitChangesLens {
private const val SHORT_HASH_LENGTH = 7
private const val PATH_SEPARATOR = "/"
private const val DATE_ATTRIBUTE = "date"
private const val AUTHOR_ATTRIBUTE = "author"
private const val CO_AUTHORS_ATTRIBUTE = "co_authors"
private const val CO_AUTHOR_SEPARATOR = "; "
private const val CHANGED_FILES_ATTRIBUTE = "changed_files"
private const val CHANGED_LINES_ATTRIBUTE = "changed_lines"
private const val COMMIT_TYPE_ATTRIBUTE = "commit_type"
private const val HOTFIX_ATTRIBUTE = "hotfix"
private const val IS_HOTFIX = 1
private const val WEIGHT_STEPS_PER_UNIT = 10_000.0
private const val SMALLEST_WEIGHT = 1 / WEIGHT_STEPS_PER_UNIT

private val ATTRIBUTE_TYPES = mapOf(
CHANGED_FILES_ATTRIBUTE to AttributeType.ABSOLUTE,
CHANGED_LINES_ATTRIBUTE to AttributeType.ABSOLUTE
CHANGED_LINES_ATTRIBUTE to AttributeType.ABSOLUTE,
HOTFIX_ATTRIBUTE to AttributeType.ABSOLUTE
)
private val ATTRIBUTE_DESCRIPTORS = mapOf(
DATE_ATTRIBUTE to descriptor("Date", "Day the commit was made, in UTC"),
AUTHOR_ATTRIBUTE to
descriptor("Author", "Who made the commit, under an invented name unless real names were asked for"),
CO_AUTHORS_ATTRIBUTE to
descriptor("Co-authors", "Who the commit names as co-authors, under invented names unless real names were asked for"),
CHANGED_FILES_ATTRIBUTE to descriptor("Files changed", "Files the whole commit changed, including those gone today"),
CHANGED_LINES_ATTRIBUTE to
descriptor("Lines changed", "Lines the whole commit added and removed, including those of files gone today")
descriptor("Lines changed", "Lines the whole commit added and removed, including those of files gone today"),
COMMIT_TYPE_ATTRIBUTE to
descriptor(
"Commit type",
"Type the subject starts with (feat, fix, docs, style, refactor, test), as the *_commits metrics count"
),
HOTFIX_ATTRIBUTE to descriptor("Hotfix", "1 when the subject names a hotfix, as the hotfix_commits metric counts it")
)

private fun descriptor(title: String, description: String) = AttributeDescriptor(
Expand All @@ -46,15 +65,22 @@ internal object CommitChangesLens {
analyzers = setOf(ANALYZER)
)

/** [linesByPathToday] holds every file at HEAD, with null for a binary one. */
fun of(commitsNewestFirst: List<LoggedCommit>, linesByPathToday: Map<String, Int?>): ClustersLens {
/**
* [linesByPathToday] holds every file at HEAD, with null for a binary one, and [peopleByHash] the names to write for each
* commit, its author first and then its co-authors.
*/
fun of(
commitsNewestFirst: List<LoggedCommit>,
linesByPathToday: Map<String, Int?>,
peopleByHash: Map<String, List<String>>
): ClustersLens {
val pathTodayByOldPath = HashMap<String, String>()
val clusters = commitsNewestFirst.mapNotNull { commit ->
val members = membersOf(commit, pathTodayByOldPath, linesByPathToday)
commit.changes.forEach { change ->
change.pathBefore?.let { pathTodayByOldPath[it] = pathTodayByOldPath[change.path] ?: change.path }
}
if (members.isEmpty()) null else clusterOf(commit, members)
if (members.isEmpty()) null else clusterOf(commit, members, peopleByHash[commit.hash].orEmpty())
}
if (clusters.isEmpty()) return ClustersLens()
val clustering = Clustering(
Expand Down Expand Up @@ -93,17 +119,24 @@ internal object CommitChangesLens {
return max(SMALLEST_WEIGHT, floor(cappedShare * WEIGHT_STEPS_PER_UNIT) / WEIGHT_STEPS_PER_UNIT)
}

private fun clusterOf(commit: LoggedCommit, members: List<ClusterMember>): Cluster = Cluster(
private fun clusterOf(commit: LoggedCommit, members: List<ClusterMember>, people: List<String>): Cluster = Cluster(
id = commit.hash,
name = "${commit.hash.take(SHORT_HASH_LENGTH)} ${commit.subject}".trim(),
members = members,
attributes = mapOf(
attributes = listOfNotNull(
DATE_ATTRIBUTE to dateOf(commit),
people.firstOrNull()?.let { AUTHOR_ATTRIBUTE to it },
coAuthorsOf(people)?.let { CO_AUTHORS_ATTRIBUTE to it },
CHANGED_FILES_ATTRIBUTE to commit.changes.size,
CHANGED_LINES_ATTRIBUTE to commit.changes.sumOf { it.changedLines ?: 0 }
)
CHANGED_LINES_ATTRIBUTE to commit.changes.sumOf { it.changedLines ?: 0 },
DefaultSemanticCommitStyle.typeOf(commit.subject)?.let { COMMIT_TYPE_ATTRIBUTE to it.name },
(HOTFIX_ATTRIBUTE to IS_HOTFIX).takeIf { HotfixCommits.isHotfix(commit.subject) }
).toMap()
)

private fun coAuthorsOf(people: List<String>): String? =
people.distinct().drop(1).takeIf { it.isNotEmpty() }?.joinToString(CO_AUTHOR_SEPARATOR)

private fun dateOf(commit: LoggedCommit): String =
Instant.ofEpochSecond(commit.committedAtEpochSecond).atOffset(ZoneOffset.UTC).toLocalDate().toString()
}
Loading
Loading