Clean up Python log analysis documentation
lint / shell-yaml-ansible (push) Failing after 20s

This commit is contained in:
Mateusz Suski
2026-05-11 17:10:10 +00:00
parent 1636f46f81
commit 8a7b7c5abc
13 changed files with 158 additions and 20 deletions
+7 -6
View File
@@ -1,6 +1,6 @@
# infra-run/scripts
This directory groups executable tooling used across the `infra-run` project. It separates shell-first operational scripts from future Python-based utilities while keeping both under one automation entry point.
This directory groups executable tooling used across the `infra-run` project. It separates shell-first operational scripts from Python-based analysis utilities while keeping both under one automation entry point.
## Diagram
@@ -9,16 +9,17 @@ flowchart TD
A["scripts"] --> B["bash"]
A --> C["python"]
B --> D["Operational toolkits"]
C --> E["Future helper utilities"]
C --> E["Analysis helper utilities"]
```
## Scope
- `bash` - current implementation area with operations toolkits.
- `python` - reserved space for future supporting utilities.
- [bash](./bash/) - operational toolkits for host health checks, disk-full triage, Veritas examples, and GPFS examples.
- [python](./python/) - read-only tools for local log parsing, reporting, and structured operational analysis.
## Notes
- The repository currently emphasizes Bash because it maps directly to day-to-day Linux operations.
- The structure leaves room for higher-level helpers without mixing concerns.
- Bash remains the right default for direct host checks and operational wrappers.
- Python is used where parsing, report generation, comparison, or JSON output is clearer than shell.
- Bash tooling should remain safe by default, readable, and validated with `../../scripts/check-bash.sh` from the repository root.
- Python tooling should remain read-only by default, standard-library based, and validated with `../../scripts/check-python.sh` from the repository root.
+67 -3
View File
@@ -1,5 +1,69 @@
# python
# Python Operational Tools
Planned area for small Python helpers.
This directory contains small Python utilities that support operational analysis in `infra-run`.
No Python tooling is implemented in `infra-run` yet.
Python is used here only when it adds practical value over Bash: parsing structured or noisy input, producing repeatable reports, comparing evidence, or emitting machine-readable output for later automation. Shell remains the default choice for direct host checks and simple command wrappers.
## Tools
| Tool | Path | Purpose | Typical use | Example command |
| --- | --- | --- | --- | --- |
| incident-log-summary | [incident-log-summary](./incident-log-summary/) | Summarize configured incident patterns from one local log file. | First-pass incident notes from system or application logs. | `python3 incident_log_summary.py --file examples/system-messages.log` |
| log-diff-checker | [log-diff-checker](./log-diff-checker/) | Compare configured patterns before and after a change. | Post-change review for new, increased, decreased, resolved, or unchanged log symptoms. | `python3 log_diff_checker.py --before examples/pre-change.log --after examples/post-change.log` |
| auth-log-audit | [auth-log-audit](./auth-log-audit/) | Summarize SSH, sudo, su, and PAM findings from local authentication logs. | Authentication incident review or access-control evidence gathering. | `python3 auth_log_audit.py --file examples/sample-auth.log` |
| jvm-log-analyzer | [jvm-log-analyzer](./jvm-log-analyzer/) | Summarize JVM exceptions, stack traces, HTTP 5xx entries, database issues, and TLS symptoms. | Java application support, restart review, or incident handoff evidence. | `python3 jvm_log_analyzer.py --file examples/sample-jvm-app.log` |
| journal-analyzer | [journal-analyzer](./journal-analyzer/) | Summarize exported `journalctl` text for failed units, restart loops, OOM events, and service warnings. | Linux service incident review or patching/change evidence. | `python3 journal_analyzer.py --file examples/sample-journal.log` |
| known-error-matcher | [known-error-matcher](./known-error-matcher/) | Match local logs against a JSON known-error catalog. | Connect known symptoms to severity, category, samples, and runbook references. | `python3 known_error_matcher.py --file examples/sample-app.log --patterns patterns.json` |
## Expected Use Cases
- Log parsing for incident review.
- Markdown or text report generation from collected evidence.
- Change evidence helpers for pre-check and post-check notes.
- Incident summary builders from sanitized inputs.
- Structured output for automation, such as JSON where useful.
## Standards
- Use the Python standard library only unless a later tool clearly justifies another dependency.
- Keep tools read-only by default.
- Do not perform destructive actions.
- Use `argparse` for command-line interfaces.
- Produce predictable text output suitable for terminal review and change notes.
- Support text, Markdown, and JSON output where useful for terminal review, tickets, or local automation.
- Use an `OK`, `WARNING`, `CRITICAL`, and `UNKNOWN` status model for findings.
- Handle malformed input, permission problems, and runtime errors defensively.
- Return meaningful exit codes.
- Keep each tool small, focused, and easy to review.
## Exit Codes
- `0` - OK, no findings, or successful validation.
- `1` - Operational findings detected.
- `2` - Invalid input, missing dependency, permission issue, or runtime error.
## Validation
From the repository root:
```bash
bash scripts/check-python.sh
bash scripts/validate-repo.sh
```
The checks use `python3 -m py_compile` and do not require external Python dependencies.
## Expected Tool Structure
Future tools should use a small self-contained layout:
```text
tool-name/
tool_name.py
README.md
examples/
sample-input.log
sample-report.md
```
Do not add package metadata, framework scaffolding, or external dependency files unless a future tool has a specific operational reason.
@@ -184,6 +184,7 @@ Review the report before attaching it. A `WARNING` or `CRITICAL` result should b
## Safety Notes
- The tool only reads the input log and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require elevated privileges unless the chosen log path requires them.
- Do not include secrets, customer data, private hostnames, or unsanitized production details in portfolio examples.
- Treat findings as prompts for operator review, not automated remediation instructions.
- Treat operational findings as prompts that require review; the tool does not prove compromise or determine root cause automatically.
@@ -153,6 +153,7 @@ Review the report before attaching it. The output is evidence for triage; it is
## Safety Notes
- The tool only reads the input log and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require elevated privileges unless the chosen log path requires them.
- Do not include secrets, customer data, private hostnames, or unsanitized production details in portfolio examples.
- Treat findings as prompts for operator review, not automated remediation instructions.
- Treat operational findings as prompts that require review; the tool does not determine root cause automatically.
@@ -209,6 +209,7 @@ Review the report before attaching it. Use it as a concise summary of exported j
## Safety Notes
- The tool only reads the input journal export and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require root privileges unless the chosen log path requires them.
- Do not include secrets, private hostnames, customer identifiers, or unsanitized production details in portfolio examples.
- Treat the output as triage evidence that requires operator review, not an automated remediation decision.
- Treat operational findings as triage evidence that requires review; the tool does not determine root cause automatically.
@@ -212,6 +212,7 @@ Review the report before attaching it. A `WARNING` or `CRITICAL` result should b
## Safety Notes
- The tool only reads the input log and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require elevated privileges unless the chosen log path requires them.
- Do not include secrets, customer data, private hostnames, tokens, or unsanitized production details in portfolio examples.
- Treat findings as prompts for operator review, not automated remediation instructions.
- Treat operational findings as prompts that require review; the tool does not determine root cause automatically.
@@ -193,6 +193,7 @@ Review the report before attaching it. A `WARNING` or `CRITICAL` result should b
## Safety Notes
- The tool only reads the input log and pattern catalog and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require elevated privileges unless the chosen log path requires them.
- Do not include secrets, private hostnames, customer identifiers, tokens, or unsanitized production details in portfolio examples.
- Treat matches as prompts for operator review, not automated remediation instructions.
- Treat operational findings as prompts that require review; the tool does not determine root cause automatically.
@@ -158,6 +158,7 @@ Use the report as a log perspective on the change. A `CRITICAL` or `WARNING` res
## Safety Notes
- The tool only reads the input logs and optionally writes a separate report.
- The implementation uses the Python standard library only and does not require package installation.
- It does not require elevated privileges unless the chosen log path requires them.
- Do not include secrets, customer data, private hostnames, or unsanitized production details in portfolio examples.
- Treat findings as prompts for operator review, not automated remediation instructions.
- Treat operational findings as prompts that require review; the tool does not determine root cause automatically.