PGL is Grafana Labs stack which is similar to ELK stack. PLG is combination of Promtail, Loki and Grafana.
P ---> Promtail
L ---> Loki
G ---> Grafana
Promtail
Promtail is independent
agent which runs in every server and send logs to Loki.
Loki
Grafana Loki is a set of components that can be
composed into a fully featured logging stack. Its log aggregation
service which collects the logs from Promtail.
Grafana
Grafana is open-source analytics and visualization
solution. It provides charts, graphs and alerts. It connects to multiple data sources and
fetch the metrics data and monitor in Grafana visualization dashboards. Its also
have ability to monitoring applications logs.
Following are steps to implements Logs Monitoring
Solution for Liferay Portal.
- Install Promtail and Configure log scraps
- Install Loki and setup Loki configuration file
- Install Grafana
- Configure Loki Data source in Grafana
- Explore logs in Grafana
Prerequisite
Install Liferay Cluster
http://www.liferaysavvy.com/2021/07/liferay-portal-apache-webserver.html
Install Promtail and Configure log scraps
Go to Grafana Loki release page and download the latest
Promtail zip file.
https://github.com/grafana/loki/releases/
Direct Link is below
https://github.com/grafana/loki/releases/download/v2.2.1/promtail-windows-amd64.exe.zip
Extract the downloaded zip file in local
Get default Promtail configuration file “promtail-local-config.yaml”
from below URL and update the scraps.
https://github.com/LiferaySavvy/liferay-logs-monitoring-plg/blob/master/promtail-local-config.yaml
OR
https://raw.githubusercontent.com/grafana/loki/v2.2.1/cmd/promtail/promtail-local-config.yaml
It’s required to create Jobs and log file location so
that Promtail will pull the logs from log files and send to Loki.
Following is configuration yaml file.
server: http_listen_port: 7060 grpc_listen_port: 0 positions: filename: /tmp/positions.yaml clients: - url: http://localhost:3100/loki/api/v1/push scrape_configs: - job_name:
grafana static_configs: - targets: - grafana labels: job: grafana __path__:
"C:/kafka-workspace/kafka-monitoring/grafana-8.0.6/data/log/grafana.log" - job_name:
liferay-node1 static_configs: - targets: - liferay-node1 labels: job: liferay-node1 __path__:
"C:/Liferay/Liferay74/liferay-ce-portal-7.4.1-ga2-node1/logs/*.log" - job_name:
liferay-node2 static_configs: - targets: - liferay-node2 labels: job: liferay-node2 __path__:
"C:/Liferay/Liferay74/liferay-ce-portal-7.4.1-ga2-node2/logs/*.log" |
We are running two nodes of Liferay cluster,
configured two jobs. It’s also required to specify Loki end point which is
running on 3100 default port.
Promtail should be run on every server and configure
the log location and file pattern in configuration yaml file.
Start Promtail service
Open command prompt and locate to Promtail root
directory and use the below command.
promtail-windows-amd64.exe --config.file=promtail-local-config.yaml |
Need to provide configuration file as input option.
Make sure service started successfully. We can see startup logs on console for
more details.
Install Loki and setup Loki configuration
file
Go to Grafana Loki release page and download the latest
Promtail zip file.
https://github.com/grafana/loki/releases/
Direct Link is below
https://github.com/grafana/loki/releases/download/v2.2.1/loki-windows-amd64.exe.zip
Extract the downloaded zip file in local
Get default Loki configuration file “loki-local-config.yaml”
from below URL.
https://github.com/LiferaySavvy/liferay-logs-monitoring-plg/blob/master/loki-local-config.yaml
OR
https://raw.githubusercontent.com/grafana/loki/v2.2.1/cmd/loki/loki-local-config.yaml
Make sure there is no port conflicts and default port 3100
used by Loki. If you change port for Loki, we must update information in Promtail
configuration file.
Loki configuration yaml file.
auth_enabled:
false server: http_listen_port: 3100 grpc_listen_port: 9096 ingester: wal: enabled: true dir: /tmp/wal lifecycler: address: 127.0.0.1 ring: kvstore: store: inmemory replication_factor: 1 final_sleep: 0s chunk_idle_period: 1h # Any chunk not receiving new logs in
this time will be flushed max_chunk_age: 1h # All chunks will be flushed when they
hit this age, default is 1h chunk_target_size: 1048576 # Loki will attempt to build chunks up to
1.5MB, flushing first if chunk_idle_period or max_chunk_age is reached first chunk_retain_period: 30s # Must be greater than index read cache
TTL if using an index cache (Default index read cache TTL is 5m) max_transfer_retries: 0 # Chunk transfers disabled schema_config: configs: - from: 2020-10-24 store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h storage_config: boltdb_shipper: active_index_directory:
/tmp/loki/boltdb-shipper-active cache_location:
/tmp/loki/boltdb-shipper-cache cache_ttl: 24h # Can be increased for faster performance
over longer query periods, uses more disk space shared_store: filesystem filesystem: directory: /tmp/loki/chunks compactor: working_directory:
/tmp/loki/boltdb-shipper-compactor shared_store: filesystem limits_config: reject_old_samples: true reject_old_samples_max_age: 168h chunk_store_config: max_look_back_period: 0s table_manager: retention_deletes_enabled: false retention_period: 0s ruler: storage: type: local local: directory: /tmp/loki/rules rule_path: /tmp/loki/rules-temp alertmanager_url: http://localhost:9093 ring: kvstore: store: inmemory enable_api: true |
Start Loki
Open command prompt and locate Loki exe file location
in local drive and use following command.
loki-windows-amd64.exe --config.file=loki-local-config.yaml |
Once service is up it can accessible on 3100
port with below URL.
|
It confirms that Loki is installed successfully.
Install Grafana
Now it’s time to install Grafana and configure the
Loki data source in Grafana.
Follow the below URL to install Grafana.
http://www.liferaysavvy.com/2021/07/grafana-installation-on-windows.html
Configure Loki Data source in Grafana
To explore logs in Grafana, need to configure the Loki
Data source.
Access Grafana with following URL
|
Go to setting --> Configuration --> Data sources
Click Add data source
Select Loki Data source
Provide Loki URL in configuration. Click on Save
and Test button then Loki data source configured in Grafana.
|
Explore logs in Grafana
Go to Grafana home page and click on Explore
and Select Loki in explore list.
Click on Log Browser --> Select Jobs --> Click on Show logs.
Now we can see Liferay logs and we can also stream the
logs.
Logs Screen: 1
Logs Screen: 2
As I website possessor I believe the content matter here is rattling great , appreciate it for your hard work. You should keep it up forever! Best of luck.
ReplyDelete섯다
Undeniably believe that which you stated. Your favorite reason appeared to be on the net the simplest thing to be aware of. I say to you, I certainly get annoyed while people consider worries that they plainly do not know about. You managed to hit the nail upon the top as well as defined out the whole thing without having side effect , people can take a signal. Will likely be back to get more. Thanks
ReplyDelete스포츠토토
Learning is the process which I think never be ended because in every stage of life a person have able to learn new things, modify and implement to his life.
ReplyDelete안전놀이터