New Data Process Created to Review/Monitor Alerts (work)

Yesterday at work, I was able to construct a data flow using SQL (via SSIS), a batch file, a Python program, and Tableau. This method will be automated (with exception of kicking off the batch file) on a weekly basis.

Essentially, all the alerts from prior week will be pulled, scrubbed, and used to create Tableau reports.

The break down.

First Process: SSIS

  • The SSIS job runs weekly, around 5 a.m. Monday mornings. I have a SQL job pulling the last 7 days of alerts from the table.
  • This data is giving in a text file, provided on a local S drive.

Second Process: Batch File

When I come into work that morning, I kick off the batch file.

  • The batch file pulls the provided file into my local PC.
  • The Python program cleans (Scrubs) the data, reformatting it to a more usable CSV file. It also applies a date/time in the filename. This way, I will always maintain a weekly data file without it overwriting.

Third Process: Tableau

  • Now I can convert the CSV file into an Excel, which is used in Tableau.
  • This allows me to construct visual reports. With these, we can track alerts to ensure a more proactive approach to issues. This also allows us to provide a view of reoccuring issues, Ect.

Matt Cole has high regard for knowledge share. He has a desire to share critical thinking and information. With a Masters in Information Technology and a wide array of certifications, while not working full-time, he wishes to knowledge share through providing insight, information organization, and critical thinking skills.

#KnowledgeShare | Matt Cole | #infobyMattCole

Similar Posts

  • Stop Compartmentalizing The Sciences

    Stop Compartmentalizing the Sciences As I explore the Zettelkasten Method, one thing continues to stand out. The relationship, connection, or association content has with each other.  What do I mean by this? As we break down the content into atomic ideas, we start seeing similarities across the differing subjects.  These similarities become compatible and give…

  • Guarding Your Lips

    Podcast: Guarding Your Lips Those who guard their lips preserve their lives, but those who speak rashly will come to ruin. Proverbs 13:3 Tweet Listen to the Episode The book of Proverbs is a well-known collection of ancient texts. Known as the Ketuvim or Writings, it consists of seven collections of wisdom materials from a variety…

  • Timeboxing Technique

    We strive to make the most out of the day. However, time tends to run, even when we do not pay attention to it. If you do not plan time out, then it will run wild. In project management, a technique often used is timeboxing. This simple technique can become a powerful tool to become…

  • Information is not Knowledge

    Information is not Knowledge “Collecting Information does not Increase your knowledge.” unknown Tweet Listen to the Episode I often hear the phrase, “I heard” or “I read somewhere.”  Information comes in different forms, through sight, sound, and experiences. However, information is not knowledge.  Knowledge is an awareness or familiarity of a fact or situation. Information…

  • What is a Data Set?

    Data sets are used all the time. Those outside of the world of metrics and computers put little thought into it. However, all processes, actions, and movement provide information in a data set. By definition a data set is a collection of information organized as a stream of bytes in logical record and block structures…