PGL is Grafana Labs stack
which is similar to ELK stack. PLG is combination of Promtail, Loki and
Grafana.
P ---> Promtail
L ---> Loki
G ---> Grafana
Promtail
Promtail is independent
agent which runs in every server and send logs to Loki.
Loki
Grafana Loki is a set of components that can be
composed into a fully featured logging stack. Its log aggregation
service which collects the logs from Promtail.
Grafana
Grafana is open-source analytics and visualization
solution. It provides charts, graphs and alerts. It connects to multiple data sources and
fetch the metrics data and monitor in Grafana visualization dashboards. Its also
have ability to monitoring applications logs.
Following are steps to implements Logs Monitoring
Solution for Liferay Portal.
- Install Promtail and Configure log scraps
- Install Loki and setup Loki configuration
file
- Install Grafana
- Configure Loki Data source in Grafana
- Explore logs in Grafana
Prerequisite
Install Liferay Cluster
http://www.liferaysavvy.com/2021/07/liferay-portal-apache-webserver.html
Install Promtail and Configure log scraps
Go to Grafana Loki release page and download the latest
Promtail zip file.
https://github.com/grafana/loki/releases/
Direct Link is below
https://github.com/grafana/loki/releases/download/v2.2.1/promtail-windows-amd64.exe.zip
Extract the downloaded zip file in local
Get default Promtail configuration file “promtail-local-config.yaml”
from below URL and update the scraps.
https://github.com/LiferaySavvy/liferay-logs-monitoring-plg/blob/master/promtail-local-config.yaml
OR
https://raw.githubusercontent.com/grafana/loki/v2.2.1/cmd/promtail/promtail-local-config.yaml
It’s required to create Jobs and log file location so
that Promtail will pull the logs from log files and send to Loki.
Following is configuration yaml file.
|
server:
http_listen_port: 7060
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name:
grafana
static_configs:
- targets:
- grafana
labels:
job: grafana
__path__:
"C:/kafka-workspace/kafka-monitoring/grafana-8.0.6/data/log/grafana.log"
- job_name:
liferay-node1
static_configs:
- targets:
- liferay-node1
labels:
job: liferay-node1
__path__:
"C:/Liferay/Liferay74/liferay-ce-portal-7.4.1-ga2-node1/logs/*.log"
- job_name:
liferay-node2
static_configs:
- targets:
- liferay-node2
labels:
job: liferay-node2
__path__:
"C:/Liferay/Liferay74/liferay-ce-portal-7.4.1-ga2-node2/logs/*.log"
|
We are running two nodes of Liferay cluster,
configured two jobs. It’s also required to specify Loki end point which is
running on 3100 default port.
Promtail should be run on every server and configure
the log location and file pattern in configuration yaml file.
Start Promtail service
Open command prompt and locate to Promtail root
directory and use the below command.
|
promtail-windows-amd64.exe
--config.file=promtail-local-config.yaml
|
Need to provide configuration file as input option.
Make sure service started successfully. We can see startup logs on console for
more details.
Install Loki and setup Loki configuration
file
Go to Grafana Loki release page and download the latest
Promtail zip file.
https://github.com/grafana/loki/releases/
Direct Link is below
https://github.com/grafana/loki/releases/download/v2.2.1/loki-windows-amd64.exe.zip
Extract the downloaded zip file in local
Get default Loki configuration file “loki-local-config.yaml”
from below URL.
https://github.com/LiferaySavvy/liferay-logs-monitoring-plg/blob/master/loki-local-config.yaml
OR
https://raw.githubusercontent.com/grafana/loki/v2.2.1/cmd/loki/loki-local-config.yaml
Make sure there is no port conflicts and default port 3100
used by Loki. If you change port for Loki, we must update information in Promtail
configuration file.
Loki configuration yaml file.
|
auth_enabled:
false
server:
http_listen_port: 3100
grpc_listen_port: 9096
ingester:
wal:
enabled: true
dir: /tmp/wal
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 1h # Any chunk not receiving new logs in
this time will be flushed
max_chunk_age: 1h # All chunks will be flushed when they
hit this age, default is 1h
chunk_target_size: 1048576 # Loki will attempt to build chunks up to
1.5MB, flushing first if chunk_idle_period or max_chunk_age is reached first
chunk_retain_period: 30s # Must be greater than index read cache
TTL if using an index cache (Default index read cache TTL is 5m)
max_transfer_retries: 0 # Chunk transfers disabled
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
storage_config:
boltdb_shipper:
active_index_directory:
/tmp/loki/boltdb-shipper-active
cache_location:
/tmp/loki/boltdb-shipper-cache
cache_ttl: 24h # Can be increased for faster performance
over longer query periods, uses more disk space
shared_store: filesystem
filesystem:
directory: /tmp/loki/chunks
compactor:
working_directory:
/tmp/loki/boltdb-shipper-compactor
shared_store: filesystem
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: false
retention_period: 0s
ruler:
storage:
type: local
local:
directory: /tmp/loki/rules
rule_path: /tmp/loki/rules-temp
alertmanager_url: http://localhost:9093
ring:
kvstore:
store: inmemory
enable_api: true
|
Start Loki
Open command prompt and locate Loki exe file location
in local drive and use following command.
|
loki-windows-amd64.exe
--config.file=loki-local-config.yaml
|
Once service is up it can accessible on 3100
port with below URL.
It confirms that Loki is installed successfully.
Install Grafana
Now it’s time to install Grafana and configure the
Loki data source in Grafana.
Follow the below URL to install Grafana.
http://www.liferaysavvy.com/2021/07/grafana-installation-on-windows.html
Configure Loki Data source in Grafana
To explore logs in Grafana, need to configure the Loki
Data source.
Access Grafana with following URL
Go to setting --> Configuration --> Data sources
Click Add data source
Select Loki Data source
Provide Loki URL in configuration. Click on Save
and Test button then Loki data source configured in Grafana.
Explore logs in Grafana
Go to Grafana home page and click on Explore
and Select Loki in explore list.
Click on Log Browser --> Select Jobs --> Click on Show logs.
Now we can see Liferay logs and we can also stream the
logs.
Logs Screen: 1
Logs Screen: 2
Author
Posts
