Fundamentals for good logging
When it comes to building your product, it’s easy for logging to fall by the wayside, but the truth is logging should never be an afterthought.
Good logging is critical to powering business intelligence and it can make all the difference in troubleshooting by providing the information needed to detect mistakes, review security measures and provide overall context on why developers made certain moves.
With that in mind, it’s important to understand the fundamentals for good logging, including the differences between basic and structured logging, uses for logging and best practices for using logs based on your desired outcomes. Let’s dive in.
Basic vs. structured logging: what’s the difference?
First, let’s cover two types of logging: Basic logging and structured logging.
Basic logging is meant to be human readable. The goal of basic logging is to make it easy for anyone to read through logs to gain insights into how the system or code operates for purposes like debugging or accessing certain information. Basic logging offers a good entry point to evaluate the health of individual software systems and pull analytics for those systems.
Structured logging applies a standard definition to how each log gets formatted. For example, this covers elements like what fields are included in the logs and impacts everything from system analyses to business outcomes. Essentially, structured logging applies a standard format for what’s included in each log that affects what you can get out of the information. Structured logging is important for sharing information with business intelligence tools.
Diving deeper, structured logging typically includes log levels to easily group different types of information. These log levels are as follows:
- Trace: Covers everything going on in the system.
- Debug: Hones in on a particular system or area of code to determine what’s going on.
- Info: Looks at activities that are expected to happen all the time but that you might want to count or keep track of to determine outputs.
- Warn: Tracks when elements enter a state that you don’t expect them to enter to provide a warning for someone to take a closer look.
- Error: Logs exceptions to understand how errors occurred, what the source of the event was and who the error affects.
- Fatal: Goes off when a system in production goes down or has a severely degraded experience.
Uses for logging: how can you apply the information?
Logs provide information that can feed into everything from executive dashboards to business intelligence tools. In most cases where you apply information from logs into other systems, it makes sense to use structured logging, as this allows you to get specific about what you want to analyze.
For instance, a common piece of information startups want to know is how many users were active in the last month. You can easily get a count for monthly active users by building a structured log that looks at when user IDs log in and then sharing that information with a business intelligence tool to track how that number trends over time and to ensure unique IDs only get counted once.
Meanwhile, at Spaceship, one of our top level metrics is customer deliveries, so a lot of our logging and analytics look at delivery trends through the Spaceship platform. This includes sharing structured log data with business intelligence tools to look at elements like increases in deliveries through the platform over time and the number of error rates per delivery.
In general, this type of relationship between a structured log and a business intelligence tool is extremely helpful for B2B SaaS companies that want to use time as a dimension for analysis.
Best practices for logging: how should you set up your logs?
Once you have a goal in mind, how should you set up your logs? There are several pieces of information that make for the most useful logs:
- Time: Time is almost always the best dimension on which to analyze things. That said, time can be tricky due to different time zones. The best practice here is to use the ISO8601 UTC timestamp for all events, that way everything gets logged in a single format and time zone. From there, you can localize the timestamp (e.g. by converting to EST) for whatever makes the most sense for your business tooling.
- Event description: You should always have a unique description of any event you log so that you can easily distinguish it from other events. This uniqueness is especially important for actions that might have multiple approaches (e.g. if your users can authenticate via email and password or via single sign-on, each of those methods should have a separate event description).
- Token action or team token action: Most B2B SaaS tools are based on either individual users or teams doing things within the system, so logging actions based on users or teams is important for everything from isolating activities to debugging to evaluating team metrics. Overall, this type of data helps you look for aggregates and averages across all of your users.
- Event source: It’s always helpful to know the source of different events that occur in the system, whether that’s a user taking action or another system communicating to trigger an action. This information helps you understand how events originated and can be helpful in determining things like whether errors were caused by user error or faulty integrations.
Of course these are just examples of many options that exist for information to include in logs. No matter what information you include, if you have structured logs, you should only have a single metric per line. Keeping each line to a single metric gives you flexibility to filter across log lines, for instance in cases where you want to filter by things like who got what message when. This structure is also important for business intelligence tools, as you can’t slice and dice the data in these tools if you have several facts presented in each log line.
Why good logging matters
Overall, good logging is fundamental to having business intelligence tools that work well. It’s also extremely helpful when it comes to debugging issues. While there’s a lot that goes into logging, what works best for your team will come down to defining the dimensions you want to analyze and then mapping that information appropriately to logs.
Interested in learning more about how good logging can help your team and what it takes to get started? Contact Spaceship today to learn how we can help.