Code, Explained

Automating SQL Server Backup Jobs with Dynamic Deployment Scripts

Managing backup jobs across multiple SQL Server instances can quickly become a repetitive administrative task. While SQL Server Agent provides a straightforward interface for creating jobs, manually configuring full, differential, and transaction log backups on every server introduces opportunities for inconsistency and human error.

A better approach is to automate the deployment of backup jobs using a configuration-driven script that creates schedules, backup commands, retention policies, and notifications automatically.

This article demonstrates how to build a reusable backup deployment framework using SQL Server Agent and dynamic T-SQL.


Why Automate Backup Deployment?

Database administrators commonly face the following challenges:

  • Different backup schedules across environments
  • Inconsistent retention settings
  • Missing operator notifications
  • Manual job creation after server migrations
  • Configuration drift over time

Automating the deployment process solves these issues by ensuring every server follows the same backup standards.


Step 1: Centralize Configuration

The first step is defining configuration values that control backup behavior.

USE msdb;
GO

DECLARE @ServiceTag sysname = 'Enterprise Protection';

DECLARE @NotificationGroup sysname =
    'Database Support Team';

DECLARE @TargetLocation nvarchar(4000) =
    '\\BackupVault\SQL\PrimaryCluster';

DECLARE @DatabaseSelection sysname =
    'USER_DATABASES';

DECLARE @BaselineKeepHours int = 96;
DECLARE @DeltaKeepHours int = 48;
DECLARE @TransactionKeepHours int = 24;

By placing settings in one location, administrators can modify retention periods, backup destinations, and database selections without changing the deployment logic.


Step 2: Validate Configuration Before Deployment

Before creating jobs, it’s important to validate configuration values.

A common safeguard is ensuring retention values are not negative and that at least one backup type is enabled.

IF @BaselineKeepHours < 0
   OR @DeltaKeepHours < 0
   OR @TransactionKeepHours < 0
   OR (@BaselineKeepHours
       + @DeltaKeepHours
       + @TransactionKeepHours) = 0
BEGIN
    THROW 51000,
          'Backup retention configuration is invalid.',
          1;
END

This prevents accidental deployment of unusable backup schedules.


Step 3: Generate Dynamic Job Names

Dynamic job names make the script reusable across multiple SQL Server environments.

DECLARE @BaselineJob nvarchar(128);
DECLARE @DeltaJob nvarchar(128);
DECLARE @TransactionJob nvarchar(128);

SET @BaselineJob =
    @ServiceTag + ' - Baseline Snapshot';

SET @DeltaJob =
    @ServiceTag + ' - Incremental Snapshot';

SET @TransactionJob =
    @ServiceTag + ' - Recovery Stream';

This approach avoids hardcoded names and improves portability.


Step 4: Ensure Idempotent Deployments

A deployment script should be safe to execute multiple times.

Before creating a job, remove any existing version.

IF EXISTS
(
    SELECT 1
    FROM msdb.dbo.sysjobs
    WHERE name = @BaselineJob
)
BEGIN
    EXEC msdb.dbo.sp_delete_job
        @job_name = @BaselineJob;
END

This guarantees the latest configuration is always deployed.


Step 5: Create a Full Backup Job

The baseline backup serves as the foundation of the recovery chain.

EXEC msdb.dbo.sp_add_job
    @job_name = @BaselineJob,
    @enabled = 1,
    @description =
    'Periodic complete database backup';

The backup command is assembled dynamically.

DECLARE @BaselineCommand nvarchar(MAX);

SET @BaselineCommand = N'
EXEC dbo.DatabaseBackup
    @Databases = ''' + @DatabaseSelection + ''',
    @Directory = ''' + @TargetLocation + ''',
    @BackupType = ''FULL'',
    @Compress = ''Y'',
    @CheckSum = ''Y'',
    @CleanupTime = '
    + CAST(@BaselineKeepHours AS nvarchar(10))
    + ',
    @DirectoryStructure =
      ''{DatabaseName}\{BackupType}'';
';

Adding the job step:

EXEC msdb.dbo.sp_add_jobstep
    @job_name = @BaselineJob,
    @step_name = 'Execute Full Backup',
    @subsystem = 'TSQL',
    @database_name = 'master',
    @command = @BaselineCommand;

Step 6: Schedule Daily Full Backups

A daily full backup schedule can be attached programmatically.

EXEC msdb.dbo.sp_add_schedule
    @schedule_name =
        'Nightly Baseline Execution',
    @freq_type = 4,
    @freq_interval = 1,
    @freq_subday_type = 1,
    @active_start_time = 010000;

Attach the schedule:

EXEC msdb.dbo.sp_attach_schedule
    @job_name = @BaselineJob,
    @schedule_name =
        'Nightly Baseline Execution';

Step 7: Deploy Differential Backups Every 6 Hours

Differential backups capture changes since the most recent full backup.

EXEC msdb.dbo.sp_add_schedule
    @schedule_name =
        'Incremental Refresh Schedule',
    @freq_type = 4,
    @freq_interval = 1,
    @freq_subday_type = 8,
    @freq_subday_interval = 6,
    @active_start_time = 000000,
    @active_end_time = 235959;

This configuration executes at:

  • 12:00 AM
  • 06:00 AM
  • 12:00 PM
  • 06:00 PM

Differential backups reduce restore times while consuming less storage than full backups.


Step 8: Configure Transaction Log Backups

Transaction log backups provide point-in-time recovery capabilities.

The schedule below executes every thirty minutes.

EXEC msdb.dbo.sp_add_schedule
    @schedule_name =
        'Continuous Recovery Schedule',
    @freq_type = 4,
    @freq_interval = 1,
    @freq_subday_type = 4,
    @freq_subday_interval = 30,
    @active_start_time = 000000;

Frequent log backups help minimize data loss and control transaction log growth.


Step 9: Register Jobs on the Server

Once jobs and schedules are created, register them with SQL Server Agent.

EXEC msdb.dbo.sp_add_jobserver
    @job_name = @BaselineJob;

EXEC msdb.dbo.sp_add_jobserver
    @job_name = @DeltaJob;

EXEC msdb.dbo.sp_add_jobserver
    @job_name = @TransactionJob;

This enables SQL Server Agent to execute the jobs automatically.


Step 10: Configure Failure Notifications

Monitoring is just as important as backups themselves.

Configure email alerts so administrators are notified when a backup job encounters an issue.

EXEC msdb.dbo.sp_update_job
    @job_name = @BaselineJob,
    @notify_level_email = 2,
    @notify_email_operator_name =
        @NotificationGroup;

The same approach can be applied to differential and transaction log backup jobs.

This ensures problems are detected immediately rather than during a recovery event.


Benefits of This Approach

Using a dynamic deployment framework provides several advantages:

  • Consistent backup standards across servers
  • Automated SQL Agent configuration
  • Reusable deployment logic
  • Simplified maintenance
  • Reduced manual effort
  • Easier disaster recovery preparation
  • Faster onboarding of new SQL Server instances

Final Thoughts

Backup jobs should be treated as infrastructure code rather than manually configured objects. By dynamically deploying full, differential, and transaction log backup jobs, organizations can standardize protection policies across all SQL Server environments.

The techniques shown in this article provide a scalable foundation for enterprise backup automation while maintaining flexibility through configuration-driven design.

Posted in Uncategorized

Reusable Python Library for KML/KMZ Route Parsing and Google Maps URL Generation

Working with GPS routes often means dealing with KML and KMZ files exported from Google Earth, Garmin devices, route planners, or GIS applications. While these formats are widely used, extracting route information and converting it into something practical—such as a Google Maps directions link—usually requires manual effort.

To simplify this process, I built a Python solution that consists of two parts:

  1. A reusable library class that can be imported into any Python application.
  2. A command-line utility that uses the library for interactive and automated workflows.

The result is a flexible tool that can parse KML and KMZ files, generate Google Maps URLs, and export location data to CSV.

Github: https://github.com/razonklnbd/kml-to-google-map-url

Why I Built This Project

This project started from a real family travel planning problem.

I was planning a long road trip for my family using Google My Maps. My goal was to create a custom route with multiple stops and then print step-by-step driving directions for my daughter so she could easily follow the journey and understand the route ahead of time.

Google My Maps is excellent for planning and visualizing custom routes, but I quickly discovered a limitation: it does not provide the same printable turn-by-turn directions that are available in the standard Google Maps interface.

One option was to manually recreate the route in Google Maps by entering every stop one by one. While this works for a route with only a few locations, it becomes tedious and time-consuming when dealing with dozens of waypoints spread across a multi-day trip.

Since Google My Maps allows routes to be exported as KML files, I started looking for a way to automatically convert that route data into a Google Maps Directions URL that could be opened directly in the standard Google Maps interface.

As a Python developer, I realized this problem was straightforward to automate:

  1. Export the route from Google My Maps as a KML or KMZ file.
  2. Extract all placemark locations.
  3. Generate a Google Maps Directions URL using those locations.
  4. Open the generated URL in Google Maps.
  5. Print the step-by-step directions from Google Maps.

What began as a small utility script for a family vacation quickly evolved into a reusable Python library and command-line tool that can parse KML/KMZ files, generate Google Maps route URLs, and export route information to CSV.

The result is a tool that saves time, eliminates manual data entry, and makes it easy to move route data from Google My Maps into standard Google Maps for navigation and printing.


Project Goals

The design goals were:

  • Support both KML and KMZ formats
  • Generate Google Maps directions URLs
  • Export route points to CSV
  • Provide both GUI and command-line workflows
  • Separate business logic from user interface code
  • Allow other developers to reuse the parsing functionality in their own projects

Architecture Overview

The project is divided into two files.

1. Library Layer

kml_route_parser.py

Contains:

class KmlRouteParser

Responsibilities:

  • Read KML files
  • Read KMZ files
  • Extract placemark points
  • Generate Google Maps URLs
  • Export CSV files
  • Provide route data programmatically

The library contains no user interface code.


2. Application Layer

main.py

Responsibilities:

  • Command-line argument processing
  • File selection dialogs
  • Save dialogs
  • User prompts
  • Displaying results

This separation follows a clean architecture approach where the reusable logic is independent of the user interface.


Supported File Formats

KML

KML (Keyhole Markup Language) is an XML-based format used to store geographic data.

Example:

<Placemark>
    <name>Home</name>
    <Point>
        <coordinates>
            72.5432,23.0345,0
        </coordinates>
    </Point>
</Placemark>

KMZ

KMZ is a compressed ZIP archive containing one or more KML files.

The library automatically:

  • Detects KMZ files
  • Extracts the embedded KML
  • Parses the route data
  • Cleans up temporary files

The user does not need to manually extract anything.


Extracting Route Points

Each placemark is converted into a structured Python dictionary.

Example:

{
    "name": "Home",
    "description": "Starting Point",
    "latitude": 23.0345,
    "longitude": 72.5432
}

The parser returns a list of points that can be consumed by any Python application.


Google Maps URL Generation

One of the primary features of the library is automatic generation of Google Maps Directions URLs.

The library supports two modes.


Name-Based URLs

Example:

https://www.google.com/maps/dir/Home/Office/Warehouse

Usage:

url = parser.get_google_maps_url(
    use_names=True
)

This produces human-readable URLs.


Coordinate-Based URLs

Example:

https://www.google.com/maps/dir/23.0345,72.5432/23.0654,72.6021

Usage:

url = parser.get_google_maps_url(
    use_names=False
)

Coordinates are often more reliable because they eliminate ambiguity caused by duplicate place names.


CSV Export

The library can export extracted points into a CSV file.

Example:

Name,Latitude,Longitude,Description
Home,23.0345,72.5432,Starting Point
Office,23.0654,72.6021,Destination

Usage:

parser.save_csv(
    "points.csv"
)

This allows route data to be easily opened in:

  • Excel
  • Google Sheets
  • Power BI
  • GIS software
  • Data analysis tools

Using the Library in Your Own Projects

The biggest advantage of the refactoring is that developers can now use the parser without the command-line application.

Example:

from kml_route_parser import KmlRouteParser

parser = KmlRouteParser(
    "route.kmz"
)

parser.load()

url = parser.get_google_maps_url(
    use_names=False
)

print(url)

Accessing Route Points Programmatically

You can access all extracted locations directly.

Example:

points = parser.get_points()

for point in points:

    print(
        point["name"],
        point["latitude"],
        point["longitude"]
    )

This makes the library suitable for:

  • GIS applications
  • Fleet management systems
  • Route optimization tools
  • Data processing pipelines
  • Custom map applications

Command-Line Utility

The project also includes a command-line application built on top of the library.

File:

main.py

Interactive File Selection

When no input file is supplied:

python main.py

a file picker dialog appears.

Supported file types:

  • KML
  • KMZ

This makes the tool accessible to non-technical users.


Command-Line File Input

Files can also be provided directly.

Example:

python main.py route.kml

or

python main.py route.kmz

This mode is useful for automation.


URL Mode Selection

If URL mode is not supplied through the command line, the application asks the user:

Build URL using:

1. Place Names
2. Coordinates

The selected mode is passed to the library.


Automated URL Mode

URL mode can be specified directly.

Use names:

python main.py route.kmz --url-mode name

Use coordinates:

python main.py route.kmz --url-mode coord

No prompt is displayed.


CSV Export Options

The application supports multiple CSV export workflows.


Save Dialog

If no CSV file is supplied, a Save As dialog appears.

The user can:

  • Choose location
  • Change filename
  • Cancel export

If cancelled:

CSV export cancelled.

No file is created.


Specify CSV File

Example:

python main.py route.kmz --csv points.csv

or

python main.py route.kmz --csv "C:\Routes\points.csv"

CSV is written directly without prompting.


Disable CSV Export

Example:

python main.py route.kmz --no-csv

The application skips CSV generation entirely.


Fully Automated Example

This example runs without any dialogs or prompts.

python main.py route.kmz \
    --url-mode coord \
    --csv points.csv

Workflow:

  1. Load KMZ file
  2. Extract route points
  3. Generate coordinate-based Google Maps URL
  4. Save CSV file

Perfect for automation scripts and scheduled jobs.


Fully Interactive Example

python main.py

Workflow:

  1. Select KML/KMZ file
  2. Choose URL generation mode
  3. Select CSV save location
  4. Receive Google Maps URL

Ideal for occasional users.


Why Split the Project into a Library and CLI?

The original version mixed:

  • Parsing logic
  • URL generation
  • User interaction
  • File dialogs

inside a single script.

By separating the code:

Benefits for Developers

  • Easier testing
  • Better maintainability
  • Cleaner architecture
  • Reusable functionality
  • Simpler future enhancements

Benefits for Users

  • Interactive desktop experience
  • Command-line automation
  • Flexible workflows

Future Enhancements

Potential improvements include:

  • GPX file support
  • Route statistics
  • Distance calculations
  • Elevation analysis
  • OpenStreetMap URL generation
  • GeoJSON export
  • Interactive map visualization
  • Batch processing of multiple files

Conclusion

By separating the project into a reusable KmlRouteParser library and a lightweight command-line application, the solution becomes useful for both developers and end users.

Developers can import the library directly into their applications, while non-technical users can continue to use graphical file dialogs and interactive prompts.

The result is a flexible tool capable of parsing KML/KMZ files, generating Google Maps route URLs, exporting CSV data, and fitting seamlessly into both manual and automated workflows.

Posted in PythonTagged , , ,