Running Sitecore XP/XC in an architecture such as Azure gives you a lot of various logs but, unfortunately, no overview. Application Insights helps, but then again, it skips data from time to time.
Both for development, and absolutely for production environments, there’s a need for a central log overview for identifying errors, help-desk and users issues, etc. Everyone has that need, so there’s multiple options available, just to name a few:
- Customize application insights, as that comes with Azure. This require some work, but it’s feasible.
- Use Splunk and other similar tools.
- Log to a local central database using “classic” log appenders from both XP and XC.
- Log to Elasticsearch using Kibana for viewing and analyzing the data.
In this post, I’ve chosen to go with the last option: using Elasticsearch and Kibana to collect and analyze logs.
THE EASE OF USING ELASTICSEARCH AND KIBANA
The nice thing about using Elasticsearch and Kibana is that it’s just so easy. You don’t have to do much. There are templates on Azure Marketplace, and if you’re just checking it out, simply follow this awesome post and you’re up running really fast: https://www.humankode.com/asp-net-core/logging-with-elasticsearch-kibana-asp-net-core-and-docker.
With this running, you have everything to need to receive data and to search and visualize it. It can’t be any simpler.
WHAT'S NEEDED ON THE SITECORE SIDE?
Since Sitecore XC is .Net Core and Sitecore XP is classic .NET, the process is a little different. Let’s look at what we did to make it work.
LOGGING TO ELASTICSEARCH FROM SITECORE XC AND .NET CORE
From Sitecore XP it’s really easy. Utilize the existing, easy-to-use NuGet packages. I ended up using Serilog.Sinks.Elasticsearch.
I implemented this in the engine making the following changes in the startup.cs in the Engine project – in the public startup method:
loggerConfig = loggerConfig.WriteTo.Elasticsearch(
new ElasticsearchSinkOptions(new Uri( Configuration.GetSection("Serilog:ElasticConfiguration:Uri").Value))
{
AutoRegisterTemplate = true,
IndexFormat = elasticIndexName,
BufferBaseFilename = $@"{Path.Combine(_hostEnv.WebRootPath, "logs")}\SerilogElasticBuffer"
}
);
Then I put the url for Elasticsearch in the config.json.
Doing this will pipe all your logging into to Elasticsearch and then you have tons of options for customizing so it fits what you would like to do, such as adding tags/categories around your logs so you can filter/facet on them in Kibana:
Log.ForContext("AuditType", "Integration").ForContext("Category", "CatalogLoad").Information(……..
“ForContext” makes it possible to add additional fields being pushed into Elasticsearch without doing anything else than that.
LOGGING TO ELASTICSEARCH FROM SITECORE XP AND CLASSIC .NET
From XP it is a bit more complex,not because of Elasticsearch, but because of Sitecore XP. Using Log4Net is really easy, but at one point, Sitecore decided to use Log4Net in an old version, put it on Sitecore.Diagnostics and still keep the original Log4Net namespace inside. That does not make it easy to work with when you add updated versions of Log4Net NuGet packages.
I managed to find some relevant source code on GitHub, modify the code to compile, and then I could use classic Log4Net loggers with my ElasticSearchAppender in Sitecore’s configs.
The nice thing about this is that we now get all logs, from both Sitecore XC and XP into the same Elasticsearch, and with Kibana we can then get a single view over all logs from all the components running.
This is crucial to any application being deployed, but it is also a must-have for any deployment and actual development. Without it, too much time is wasted on downloading and searching a lot of different log files across too many applications.
I hope you found this useful! Reachout to me if you need additional hints to get your project moving.
ROLAND VILLEMOES
CTO
