New Data Process Created to Review/Monitor Alerts (work)

ByMatt

Yesterday at work, I was able to construct a data flow using SQL (via SSIS), a batch file, a Python program, and Tableau. This method will be automated (with exception of kicking off the batch file) on a weekly basis.

Essentially, all the alerts from prior week will be pulled, scrubbed, and used to create Tableau reports.

The break down.

First Process: SSIS

  • The SSIS job runs weekly, around 5 a.m. Monday mornings. I have a SQL job pulling the last 7 days of alerts from the table.
  • This data is giving in a text file, provided on a local S drive.

Second Process: Batch File

When I come into work that morning, I kick off the batch file.

  • The batch file pulls the provided file into my local PC.
  • The Python program cleans (Scrubs) the data, reformatting it to a more usable CSV file. It also applies a date/time in the filename. This way, I will always maintain a weekly data file without it overwriting.

Third Process: Tableau

  • Now I can convert the CSV file into an Excel, which is used in Tableau.
  • This allows me to construct visual reports. With these, we can track alerts to ensure a more proactive approach to issues. This also allows us to provide a view of reoccuring issues, Ect.

Matt Cole has high regard for knowledge share. He has a desire to share critical thinking and information. With a Masters in Information Technology and a wide array of certifications, while not working full-time, he wishes to knowledge share through providing insight, information organization, and critical thinking skills.

#KnowledgeShare | Matt Cole | #infobyMattCole

Similar Posts

  • The Pomodoro Technique

    ByMatt

    Life continues to become complicated, with many streams coming at you. All in order to take your most precious commodity, time. I want to introduce the Pomodoro Technique. I unique, simple method to help you focus on a task, with a burst of hyper-focus. The best part, it requires no apps, item purchase, or unrealistic…

  • Critical Thinking

    ByMatt

    Admittedly, I am pretty harsh on the up coming generation. I find they appear to lose the ability to critically think, have no knowledge of history, and seems to focus on the ‘now’ as well as ‘quick fixes.’ However, the fault lies upon the prior generation, for not giving them the skills to overcome. With…

  • Stoicism and Neuro-Linguistic Programming (NLP)

    ByMatt

    Stoicism and Neuro-Linguistic Programming (NLP) are two areas of study that may seem quite different on the surface, but they share some interesting similarities and differences. Both are focused on developing inner strength and resilience, but they approach this goal in different ways. Stoicism is a philosophy that originated in ancient Greece and emphasizes the…

  • Personal Information Bank

    ByMatt

    Personal Information Bank What I need is to make my own personal information bank. Over time, I realized I seem to have forgotten more than I currently know. This is done due to information changing, advancing, growing and at times being discarded or inaccurate.  What I should had been doing is creating a personal knowledge…

  • When we Pivot

    ByMatt

    Pivot is the act of turning or a person or thing on which something depends; the central or crucial factor. I mentioned the need to pivot in another article I wrote. What does it mean to Pivot? When something isn’t working, you need to pivot (change) to something else. Basically, it is a switch to…