New Data Process Created to Review/Monitor Alerts (work)

ByMatt

Yesterday at work, I was able to construct a data flow using SQL (via SSIS), a batch file, a Python program, and Tableau. This method will be automated (with exception of kicking off the batch file) on a weekly basis.

Essentially, all the alerts from prior week will be pulled, scrubbed, and used to create Tableau reports.

The break down.

First Process: SSIS

  • The SSIS job runs weekly, around 5 a.m. Monday mornings. I have a SQL job pulling the last 7 days of alerts from the table.
  • This data is giving in a text file, provided on a local S drive.

Second Process: Batch File

When I come into work that morning, I kick off the batch file.

  • The batch file pulls the provided file into my local PC.
  • The Python program cleans (Scrubs) the data, reformatting it to a more usable CSV file. It also applies a date/time in the filename. This way, I will always maintain a weekly data file without it overwriting.

Third Process: Tableau

  • Now I can convert the CSV file into an Excel, which is used in Tableau.
  • This allows me to construct visual reports. With these, we can track alerts to ensure a more proactive approach to issues. This also allows us to provide a view of reoccuring issues, Ect.

Matt Cole has high regard for knowledge share. He has a desire to share critical thinking and information. With a Masters in Information Technology and a wide array of certifications, while not working full-time, he wishes to knowledge share through providing insight, information organization, and critical thinking skills.

#KnowledgeShare | Matt Cole | #infobyMattCole

Similar Posts

  • Stoicism and Neuro-Linguistic Programming (NLP)

    ByMatt

    Stoicism and Neuro-Linguistic Programming (NLP) are two areas of study that may seem quite different on the surface, but they share some interesting similarities and differences. Both are focused on developing inner strength and resilience, but they approach this goal in different ways. Stoicism is a philosophy that originated in ancient Greece and emphasizes the…

  • Learn from History

    ByMatt

    Learn From History What men learn from history is that men never learn from history. Dr. Gene Kim Tweet Listen to the Episode When I was in public school, I really didn’t have a keen interest in history. I thought it was interesting, but unlike today, I find history not only fascinating, but holds a…

  • Cause of Failure

    ByMatt

    Cause of Failure Action without planning is the cause of every failure. Alex MacKenzie Tweet Listen to the Episode There are many factoids presenting planning as the action to reduce time, increase efficiency as well as providing success. Such facts used are;  For each moment spent in planning, ten minutes of execution is saved. 10/90…

  • Adapt Your Environment

    ByMatt

    Adapt Your Environment We have to adapt to you, you don’t adapt to your environment. Cassandra Aarssen Tweet Listen to the Episode Over the weekend, I was watching HGTV’s show Hot Mess House. Cassandra Aarssen, who goes by Cass, helps identify the person’s organization type, then provides the environment and tools to assist them in…

  • The Power of a Note-Taker

    ByMatt

    Let’s face it, we cannot recall everything. Sure there are those out there providing methods to memory recall. However, if you learning something new, or have that awesome nugget of information come your way, what better way than write it down for later. Writing next week’s newsletter, I came across some pretty solid evidence note-taking…