Log File in Python - codingem.com

Log to File in Python — Logging messages to a File in Python. Image design by Elias Ervast

There is a built-in module called logging for creating a log file in Python.

To log a message into a separate text file by configuring the logging module’s root logger and logging a debug message:

import logging

logging.basicConfig(filename="log.txt", level=logging.DEBUG)
logging.debug("Debug logging test...")

As a result, you see a file called log.txt in the same folder with your program. This file should have a message that looks like this:

DEBUG:root:Debug logging test...

In Python, you can log information about issues in a separate file. This file can later help you diagnose issues. It leaves a trail of breadcrumbs that leads to the source of the problem.

In this comprehensive guide, you learn how to work with the logging module and how to set up a logger.

Logging in Python—A Detailed Tour

Let’s take a more detailed tour of logging and log files in Python.

Logging means tracking the events and the status of your application as it runs. It is a crucial method to develop, debug, and run the software.

If you do not keep the track of any logs of your program, chances are you cannot diagnose issues properly. This is because without logging there is no trail that leads to the possible root cause of a problem.

With proper logging set up, you can follow the trail of breadcrumbs to the source of a problem.

Why No Printing

While printing is an excellent strategy to debug code, it may not always be the best option. This is especially true when the program is more complex.

The problem with printing is that the prints are not stored anywhere. If your application throws an error, there is no trace of that error—unless the error message is logged somewhere.

So you may want to log the data of your program in a separate log file.

In Python, there is a built-in library called logging for this purpose.

How to Start Logging Messages in Python

To start logging messages to a file, you need to know how to set up a logger.

To do this, you need to

Import the logging module
Configure the logger using the basicConfig() method. The rest of the steps describe how.
Specify the file to which log messages are sent.
Define the “seriousness” level of the log messages.
Format the log messages.
Append or overwrite previous log messages in the file.

Let’s start building a logger while going through these six steps in detail.

1. Import the Python’s Built-In Logging Module

To use Python’s built-in logging functionality, start by importing the logging module.

import logging

2. Call basicConfig() to Start Configuring the Logger.

To start logging messages to a file in Python, you need to configure the logging module.

To configure the logging module, call logging.basicConfig():

import logging

logging.basicConfig()

An empty basicConfig() call does nothing useful. In the following steps, you are going to add parameters to this call to actually configure the logger.

3. Log Target File in Python

The basicConfig() method accepts a parameter called filename. This is a string that specifies the target file for log messages.

For example, let’s start logging the messages into a file called log.txt.

import logging

logging.basicConfig(filename="log.txt")

Now the logger knows the target file and can already start logging messages into it.

But before we log our first message, we need to specify the “seriousness” level of the messages we want to track.

Let’s have a look at the log message levels in Python.

4. Log Message Levels

There are five main levels of debug messages when logging. These levels describe the “seriousness” of the issue.

The message levels are:

DEBUG. Used to give detailed information. This level is mostly used for diagnosing issues in code.
INFO. Confirms the program works as expected.
WARNING. An indication that an unexpected event occured or may occur.
ERROR. Serious issue. Indicates that a program was unable to perform some action due to an error.
CRITICAL. A serious error. Indicates that the program may be unable to continue running.

The logging levels are integers behind the scenes. Here is a table

Level	Numeric Value
`CRITICAL`	50
`ERROR`	40
`WARNING`	30
`INFO`	20
`DEBUG`	10

Logging levels according to the logging module.

When you specify a logging level, the program logs messages from that level up.

For example, if you use logging.WARNING, only messages of level WARNING, ERROR, CRITICAL are logged.

To log a message at a specific level, use the built-in logging methods that correspond to the levels specified above.

Here are the logging methods:

logging.debug(message). Logs a message on a DEBUG level.
logging.info(message). Logs a message on an INFO level.
logging.warning(message). Logs a message on an WARNING level.
logging.error(message). Logs a message on a ERROR level.
logging.critical(message). Logs a message on a CRITICAL level.

Let’s head back to configure our logger. Let’s specify the message level as DEBUG to log all the possible messages into the file:

import logging

logging.basicConfig(filename="log.txt", level=logging.DEBUG)

Now you can log your first message into the log.txt file. To do this, add the following line after the basicConfig() call:

logging.debug("Debug logging test...")

Then, run the program and see that a new file called log.txt appears in your project’s folder. Open up the file and you should see this message in the file:

DEBUG:root:Debug logging test...

And there you have it. Now that your log message level is set DEBUG, you can use all the other logging methods to log more “serious” errors. Here is how the whole program may look like now:

import logging

logging.basicConfig(filename="log.txt", level=logging.DEBUG)

logging.debug("Debug logging test...")
logging.info("Program is working as expected")
logging.warning("Warning, the program may not function properly")
logging.error("The program encountered an error")
logging.critical("The program crashed")

Running this piece of code logs all the different levels of messages to the log.txt file, which looks like this:

DEBUG:root:Debug logging test...
DEBUG:root:Debug logging test...
INFO:root:Program is working as expected
WARNING:root:Warning, the program may not function properly
ERROR:root:The program encountered an error
CRITICAL:root:The program crashed

5. How to Format Log Messages

Now all the messages are formatted in this way:

LEVEL:logger:Message

Here:

The first element is the message level. For example DEBUG.
The second element is the logger object. In this guide, we use the root logger, so it shows root.
The third item is the logged message.

But the formatting could include something more informative, such as the time and message.

To get rid of the default formatting, you can customize the format by specifying the format key in the basicConfig() call.

For example, let’s format the logged messages such that only the time and message are shown. Let’s modify the basicConfig() call:

logging.basicConfig(filename="log.txt", level=logging.DEBUG,
                    format="%(asctime)s %(message)s")

Now you can run the whole program again:

import logging

logging.basicConfig(filename="log.txt", level=logging.DEBUG,
                    format="%(asctime)s %(message)s")

logging.debug("Logging test...")
logging.info("The program is working as expected")
logging.warning("The program may not function properly")
logging.error("The program encountered an error")
logging.critical("The program crashed")

Now the output is formatted with the timestamp just like we wanted:

2021-09-28 19:42:54,461 Logging test...
2021-09-28 19:42:54,461 The program is working as expected
2021-09-28 19:42:54,461 The program may not function properly
2021-09-28 19:42:54,461 The program encountered an error
2021-09-28 19:42:54,461 The program crashed

(The integer 461 is the number of milliseconds this operation took.)

Notice that this is not the only info you can include in the logged message. Here is a full list of all the information you can show about the message.

6. Append or Overwrite the Log File in Python

Try running the example logging program multiple times. If you open up the file, you can see the errors being appended to the file. The logs from the previous runs are still there.

This is because, by default, the logger is configured to append messages to the log file.

You can also change this by specifying the filemode in the basicConfig() call.

If you want to overwrite the logs instead of appending them, specify the filemode as “w”:

logging.basicConfig(filename="log.txt", level=logging.DEBUG,
                    format="%(asctime)s %(message)s", filemode="w")

If you now run the program, the new logs overwrite the previous ones.

One last time here is the full program:

import logging

logging.basicConfig(filename="log.txt", level=logging.DEBUG,
                    format="%(asctime)s %(message)s", filemode="w")

logging.debug("Logging test...")
logging.info("The program is working as expected")
logging.warning("The program may not function properly")
logging.error("The program encountered an error")
logging.critical("The program crashed")

Conclusion

Today you learned how to create a log file in Python and how to log messages of different levels to it.

A log file can be useful to track errors and problems in your code. A properly configured log file leads to the source of an error quite easily.

Thanks for reading. I hope you find it useful.

Happy coding!

Log to a File in Python