Do you even need config files?
If a developer has a module in their project that connects with an FTP server to download some files, then the developer would write a method to connect to FTP via FTP URL and use credentials like username and password for successful connection. If the developer uses these credentials in the code and hardcodes them in their code file and deploy the application, it could work fine, job done! Now, imagine that after two months the password for that FTP site is changed and the developer must again update that password in your application to make sure it does not break the existing functionality of FTP connection. Now in this scenario, the developer would again change the code and redeploy it. Here, a developer must for a small configuration change, take the latest of the code, do the required change, make sure nothing else breaks, re-deploy the application and then test it. And this could be a repetitive task after 2~3 months when some configuration again changes. What if the application has a configuration file that has key value pairs as “User”: “<username>”, “Password”: “password” and whenever some change is needed only that file is touched and configurations are updated rather than digging into the actual code.
Why config files?
Coming from a .Net background, I found it a bit challenging in figuring out how a developer can have a configuration file in the Python application which could be used to read setting values and one does not have to even touch the code to update or save settings. Config files are used to store key value pairs or some configurable information that could be read or accessed in the code and at some point, of time. If that configuration changes, developers can just change the configuration in that config file and not worry about changing the code as it may require re-compiling the code and deploying it. Not only this but using config files makes your settings and code more reusable and keep the settings information at a centralized location and segregated. Of course, sensitive information like passwords, secrets, and certificates should be kept more secure, maybe in cloud vaults. But basic settings used in the application could be part of a configuration file.
Microsoft.Net vs Python config files
Microsoft .Net applications sometimes by default provide files like appSettings, web.config, app.config when you start with a project template, based on what kind of application you are creating. These files serve the purpose of storing settings information.
Python applications do not by default provide a settings file, however, you can use an existing one and modify it. But the best way is to create your own from scratch and use it as needed.
Getting Started
In this article, we’ll have a walkthrough of how to create a config file, add a configuration, update a configuration, delete a configuration, and read a configuration in a python application.
We’ll use ConfigParser module to deal with config files and see how easy it could be to generate and read configuration files. Python can have config files with all settings needed by the application dynamically or periodically. Python config files have the extension as .ini.
We’ll use VS Code (Visual Studio Code) to create a main method that uses config file to read the configurations and then print on the console. This could be very new to a developer like me who has just started working on Python, so, we’ll start from scratch.
Pre-requisite
I’ll not go into details of installing Python and configuring VS Code to run a python application and assume that you have it. If not, I can post a separate article based on a request on how to get started with Python in VS Code
Create a Config File
Launch VS Code and create a main.py file
Open VS Code and create a new file and name it main.py
Start from Hardcoding.
Write a small code that reads hardcoded values and print those. To start with and for the sake of understandingI am first using hard coded values to print then later will fetch those from config file.
ftpUrl = "demoftp.codeteddy.com"userName = "codeteddy"password = "my#supersecret#password"print("\nDisplaying FTP details\n")print("FTP URL: " + ftpUrl)print("FTP User Name: " + userName)print("Password: " + password)
Run the code.
Now run the code to print the details. At the corner, there should be an icon to run your code and display the output on terminal.
So, the output is,
Displaying FTP details
FTP URL: demoftp.codeteddy.comFTP User Name: codeteddyPassword: my#supersecret#passwordPS D:\Articles\Python_ConfigFile\Code>Our goal is to make sure that the output remains same, but the values of FTP settings are read from a config file.
Code to generate config file.
In VS Code, create a new file named generate_config.py and import configparser module as shown below,
import configparser
Now put the following code into the file,
import configparser# CREATE OBJECTconfig_file = configparser.ConfigParser()# ADD SECTIONconfig_file.add_section("FTPSettings")# ADD SETTINGS TO SECTIONconfig_file.set("FTPSettings", "ftpUrl", "demoftp.codeteddy.com")config_file.set("FTPSettings", "userName", "codeteddy")config_file.set("FTPSettings", "password", "my#supersecret#password")# SAVE CONFIG FILEwith open(r"configurations.ini", 'w') as configfileObj: config_file.write(configfileObj) configfileObj.flush() configfileObj.close()print("Config file 'configurations.ini' created")# PRINT FILE CONTENTread_file = open("configurations.ini", "r")content = read_file.read()print("Content of the config file are:\n")print(content)read_file.flush()read_file.close()So, we import configparser module that contains methods and classes which could be used to define a layout of the config file that we need. We create an object of Configparser and name it config_file. A config file can have sections against which the details would be stored i.e., FTP Settings or Logger settings, etc. So, we created one section named “FTPSettings” and added key value pairs to that section via config_file.set method which takes first argument as section name, second as key and third as value. In our case section name is “FTPSettings” and three keys would be ftpUrl, userName and password.
We then save the layout defined in an ini file named configurations.ini. You can give the file name of your choice where the settings would be saved. Then at last we are retrieving the content of the saved ini file and printing those on the terminal window or console.
Once you save this file and hit on run, it will generate a configurations.ini file.
Generated file.
If we open the configurations.ini file i.e., the newly generated file, we see the following settings in place,
[FTPSettings]ftpurl = demoftp.codeteddy.comusername = codeteddypassword = my#supersecret#passwordIt contains the section FTPSettings and our FTP settings as key value pairs under it. Each time the code for generate_config is run, it will create a fresh config file with the layout as defined in the code file.
Let’s add one more section to the file. But this time we’ll add it in a different way. For e.g., if we need to add a section for Logger and its corresponding settings, we can also use the following code to do so in a single code statement,
# ADD NEW SECTION AND SETTINGSconfig_file["Logger"]={ "LogFilePath":"<Path to log file>", "LogFileName" : "<Name of log file>", "LogLevel" : "Info" }Now, the generate_config.py looks like as follows with the newly added section,
import configparser# CREATE OBJECTconfig_file = configparser.ConfigParser()# ADD FTPSettings SECTIONconfig_file.add_section("FTPSettings")# ADD SETTINGS TO FTPSettings SECTIONconfig_file.set("FTPSettings", "ftpUrl", "demoftp.codeteddy.com")config_file.set("FTPSettings", "userName", "codeteddy")config_file.set("FTPSettings", "password", "my#supersecret#password")# ADD NEW SECTION AND SETTINGSconfig_file["Logger"]={ "LogFilePath":"<Path to log file>", "LogFileName" : "<Name of log file>", "LogLevel" : "Info" }# SAVE CONFIG FILEwith open(r"configurations.ini", 'w') as configfileObj: config_file.write(configfileObj) configfileObj.flush() configfileObj.close()print("Config file 'configurations.ini' created")# PRINT FILE CONTENTread_file = open("configurations.ini", "r")content = read_file.read()print("Content of the config file are:\n")print(content)read_file.flush()read_file.close()When we run the code, we see that our config file is updated with these new details,
[FTPSettings]ftpurl = demoftp.codeteddy.comusername = codeteddypassword = my#supersecret#password[Logger]logfilepath = <Path to log file>logfilename = <Name of log file>loglevel = InfoSo, we can use either of the ways i.e. add_section() and set method or object initializer way to create section and add settings to it.
Adding/Updating/Deleting configuration file settings
We can add or update or delete the settings in the config file via code as well.
Adding/Updating settings in the config file
For e.g. If we need to update the “loglevel” from “Info” to “Debug”, or we need to add a new setting to the Logger section. Either you can update the setting in the same generate_config.py file and run it or you can write code to read the existing configuration file and update new settings. For e.g. I created a new python file named update_config.py and added following code to it,
import configparser# CREATE OBJECTconfig_file = configparser.ConfigParser()# READ CONFIG FILEconfig_file.read("configurations.ini") # UPDATE A FIELD VALUEconfig_file["Logger"]["LogLevel"]="Debug" # ADD A NEW FIELD UNDER A SECTIONconfig_file["Logger"].update({"Format":"(message)"}) # SAVE THE SETTINGS TO THE FILEwith open("configurations.ini","w") as file_object: config_file.write(file_object) # DISPLAY UPDATED SAVED SETTINGSprint("Config file 'configurations.ini' is updated")print("Updated file settings are:\n")file=open("configurations.ini","r")settings=file.read()print(settings)Here in this code, we first import, create the object of configparser. We read the existing configuration file and then set the LogLevel from Logger section to “Debug”. Likewise, we added one more setting under Logger section via update method by this code:
config_file["Logger"].update({"Format":"(message)"})This code adds a new setting named “Format” with value as “(message)” under Logger section. In the end, we again save the file and then again read it to display the saved information.
Now when you run the code, the output is
Config file 'configurations.ini' is updated.
Updated file settings are,
[FTPSettings]ftpurl = demoftp.codeteddy.comusername = codeteddypassword = my#supersecret#password[Logger]logfilepath = <Path to log file>logfilename = <Name of log file>loglevel = Debugformat = (message)
So, these are the ways, you can update the ini file.
Deleting the settings
We can remove the settings from config files usingremove_option()andremove_section()module in configparser module.remove_option()is used to delete a field (i.e., key value pair) from any section andremove_section()is used to remove a complete section of the config file.
# DELETE A FIELD IN THE SECTIONconfig_file.remove_option('Logger', 'Format') # DELETE A SECTIONconfig_file.remove_section('Logger')Reading the configuration file in Python
Now that the file is created, and we have learned how to generate and update the config file when needed, let’s visit the last section of this article i.e., reading the values from config file in our main class. This is the most simple and straightforward part.
Let’s make it a bit organized and reusable. Let’s add a helper file in our VS Code workspace and name it helper.py and add the following code to it,
import configparser# Method to read config file settingsdef read_config(): config = configparser.ConfigParser() config.read('configurations.ini') return config
Now this helper method code will serve as reusable method wherever we need the configurations.
Now. Go to the main.py file and import this helper file as
Now, in the main.py, replace the hardcoded pieces with reading the settings from config file as,
import helperconfig = helper.read_config()ftpUrl = config['FTPSettings']['ftpUrl']userName = config['FTPSettings']['userName']password = config['FTPSettings']['password']print("\nDisplaying FTP details\n")print("FTP URL: " + ftpUrl)print("FTP User Name: " + userName)print("Password: " + password)And run the code. We’ll see that our output is same as when it was reading the values from the hardcoded code.
Now it reads from the config file.
