Initial CV-aligned infrastructure portfolio

Rework portfolio around Linux operations, Zabbix monitoring, migration validation, and ELK/Grafana log observability. Add AAP-style LVM resize workflow, Zabbix server/proxy/agent automation assets, Linux/AIX monitoring templates, and updated validation CI.
2026-05-04 17:37:24 +00:00
commit 35e6b139fc
114 changed files with 6422 additions and 0 deletions
@@ -0,0 +1,98 @@
+# Log Observability ELK/Grafana
+
+## Problem
+
+Operations teams need searchable logs and reviewable incident evidence in addition to simple OS checks. Zabbix is useful for host and service health signals; ELK/Grafana is better suited for log ingestion, error analysis, dashboards, and environment-level observability.
+
+## CV Relevance
+
+This project supports the monitoring and troubleshooting part of the CV by showing how incident logs can be collected, parsed, searched, and reviewed. It is separate from the Zabbix project: Zabbix handles simple checks, while this project focuses on logs and observability evidence.
+
+## What This Project Demonstrates
+
+- A local Docker Compose scaffold for Elasticsearch, Logstash, Kibana, Grafana, and Filebeat.
+- Minimal configs required for the stack to validate independently.
+- Sample logs and alert intent that can be reviewed without starting the full stack.
+- An incident simulation script for generating operational log evidence.
+
+This is a local demo stack. The default credentials are for non-production use only.
+
+## Architecture
+
+```
+Application/System Logs -> Filebeat -> Logstash -> Elasticsearch -> Kibana
+                                                       |
+                                                       v
+                                                    Grafana
+
+Incident Scenario -> Sample Logs -> Alert Rules -> Operator Review
+```
+
+Core components:
+
+- `docker-compose.yml` defines the observability services.
+- `alerting/alert_rules.yml` records alert intent and severity.
+- `examples/` contains representative operational logs and alert output.
+- `scenarios/incident_simulation.sh` emits incident activity.
+- `grafana/`, `kibana/`, `logstash/`, `filebeat/`, and `elasticsearch/` contain minimal local configs.
+
+## Quickstart
+
+```bash
+cd professional-infra/log-observability-elk-grafana
+make test
+make demo
+```
+
+Start the full local stack with Docker:
+
+```bash
+make test
+make run
+make down
+```
+
+When running locally:
+
+- Kibana: `http://localhost:5601`
+- Grafana: `http://localhost:3000`
+- Elasticsearch: `http://localhost:9200`
+
+Default demo credentials:
+
+- Elasticsearch/Kibana: `elastic` / `elastic`
+- Grafana: `admin` / `admin`
+
+## Validation
+
+```bash
+make test
+docker compose config --quiet
+```
+
+`make test` also checks that all bind-mounted config files and directories exist.
+
+## Example Output
+
+```text
+[2026-04-29 04:18:23] WARN Database connection pool nearing capacity
+[2026-04-29 04:18:28] ERROR Database connection pool exhausted
+[2026-04-29 04:18:33] ERROR Database query timeout occurred
+[2026-04-29 04:18:44] INFO Database connections restored
+```
+
+Additional examples are available in [examples/alert-output.txt](examples/alert-output.txt) and [examples/sample-log.txt](examples/sample-log.txt).
+
+## Interview Talking Points
+
+- When to use Zabbix checks versus ELK log analysis.
+- How Filebeat, Logstash, and Elasticsearch fit into a basic log pipeline.
+- How incident simulations create evidence for troubleshooting discussions.
+- Why local demo credentials and single-node Elasticsearch are not production architecture.
+
+## Roadmap
+
+- Add curated Grafana and Kibana dashboards.
+- Add Prometheus metrics collection.
+- Add distributed tracing with Jaeger or OpenTelemetry.
+- Add synthetic monitoring checks.