New Data Process Created to Review/Monitor Alerts (work)

ByMatt

Yesterday at work, I was able to construct a data flow using SQL (via SSIS), a batch file, a Python program, and Tableau. This method will be automated (with exception of kicking off the batch file) on a weekly basis.

Essentially, all the alerts from prior week will be pulled, scrubbed, and used to create Tableau reports.

The break down.

First Process: SSIS

  • The SSIS job runs weekly, around 5 a.m. Monday mornings. I have a SQL job pulling the last 7 days of alerts from the table.
  • This data is giving in a text file, provided on a local S drive.

Second Process: Batch File

When I come into work that morning, I kick off the batch file.

  • The batch file pulls the provided file into my local PC.
  • The Python program cleans (Scrubs) the data, reformatting it to a more usable CSV file. It also applies a date/time in the filename. This way, I will always maintain a weekly data file without it overwriting.

Third Process: Tableau

  • Now I can convert the CSV file into an Excel, which is used in Tableau.
  • This allows me to construct visual reports. With these, we can track alerts to ensure a more proactive approach to issues. This also allows us to provide a view of reoccuring issues, Ect.

Matt Cole has high regard for knowledge share. He has a desire to share critical thinking and information. With a Masters in Information Technology and a wide array of certifications, while not working full-time, he wishes to knowledge share through providing insight, information organization, and critical thinking skills.

#KnowledgeShare | Matt Cole | #infobyMattCole

Similar Posts

  • Fallacies or Errors in Logic

    ByMatt

    Logic is of the Stoic mindset, and a high regard involving critical thinking. However, errors in logic are called fallacies, and can damage one’s respective view and argument. Check out my posts ‘What is Critical Thinking‘ and ‘Critical Thinking.’  More sinister logic fallacies are at times purposedly used as reasonable statements. These purposeful fallacies are…

  • Gathering Data

    ByMatt

    Gathering data is a critical aspect of project planning that helps project managers make informed decisions and ensure project success. Data collection provides valuable insights into project performance, identifies potential risks and issues, and facilitates continuous improvement. In this article, we will discuss the importance of gathering data in project planning and provide tips for…

  • Machine Learning

    ByMatt

    Machine Learning Machine learning is the use and development of computer systems that are able to learn and adapt without following explicit instructions, by using algorithms and statistical models to analyze and draw inferences from patterns in data Machine learning is artificial intelligence. However, artificial intelligence is not machine learning. What makes up Machine Learning?…

  • Decline Reading Proficiency Continues for Kids

    ByMatt

    It is estimated that 67% of fourth-grade students do not read proficiently at their grade level. Only one-third of high school seniors are considered proficient readers, and only 4% of seniors reached the highest level of advanced proficiency in reading on a recent test. In fact, only 39% of teacher preparation programs across the nation…