New Data Process Created to Review/Monitor Alerts (work)

ByMatt

Yesterday at work, I was able to construct a data flow using SQL (via SSIS), a batch file, a Python program, and Tableau. This method will be automated (with exception of kicking off the batch file) on a weekly basis.

Essentially, all the alerts from prior week will be pulled, scrubbed, and used to create Tableau reports.

The break down.

First Process: SSIS

  • The SSIS job runs weekly, around 5 a.m. Monday mornings. I have a SQL job pulling the last 7 days of alerts from the table.
  • This data is giving in a text file, provided on a local S drive.

Second Process: Batch File

When I come into work that morning, I kick off the batch file.

  • The batch file pulls the provided file into my local PC.
  • The Python program cleans (Scrubs) the data, reformatting it to a more usable CSV file. It also applies a date/time in the filename. This way, I will always maintain a weekly data file without it overwriting.

Third Process: Tableau

  • Now I can convert the CSV file into an Excel, which is used in Tableau.
  • This allows me to construct visual reports. With these, we can track alerts to ensure a more proactive approach to issues. This also allows us to provide a view of reoccuring issues, Ect.

Matt Cole has high regard for knowledge share. He has a desire to share critical thinking and information. With a Masters in Information Technology and a wide array of certifications, while not working full-time, he wishes to knowledge share through providing insight, information organization, and critical thinking skills.

#KnowledgeShare | Matt Cole | #infobyMattCole

Similar Posts

  • Enhance Project Outcomes

    ByMatt

    Enhancing project outcomes is a key objective for any project manager. The success of a project is measured not only by its ability to meet the project goals but also by how well it meets or exceeds the expectations of stakeholders. In this article, we will discuss strategies for enhancing project outcomes. Recently, my role has changed…

  • Johann Adam Weishaupt BluePrint

    ByMatt

    With any form of project, one must have a blueprint consisting of specific tasks and controlled reactions. The founder of the Illuminati Johann Adam Weishaupt had also comprised structure in the forming of the elusive Illuminati group while infiltrating into other organizations. Let’s define what this blueprint consists of, and if we see this occurring….

  • Digital Identity and Central Bank Digital Currencies: The Architecture of Global Financial Surveillance

    ByMatt

    Introduction: The Convergence of Digital Identity and Financial Control The global financial landscape stands at a critical juncture, with coordinated international efforts rapidly advancing toward the implementation of comprehensive digital identity systems integrated with Central Bank Digital Currencies (CBDCs). This convergence represents far more than technological innovation—it constitutes the foundational architecture for unprecedented surveillance and…

  • Candle

    ByMatt

    Candle A candle loses nothing by lighting another candle. Father James Keller Tweet Listen to the Episode In today’s neoliberalism structure, the strong survive, and rarely share among its competitors. While it is nice to share, it has become a trait lost as we grow older. Interestingly, a national survey showed 43% of parents believed…

  • What is a Data Set?

    ByMatt

    Data sets are used all the time. Those outside of the world of metrics and computers put little thought into it. However, all processes, actions, and movement provide information in a data set. By definition a data set is a collection of information organized as a stream of bytes in logical record and block structures…

  • Knowledge is to Share

    ByMatt

    Knowledge is to Share “Knowledge is the new rich, arm yourselves with it. ” Tomi Payne Tweet Listen to the Episode I conveyed more than once, knowledge is not simply information. Knowledge goes beyond having information for the sake of instruction, advice or intelligence. Knowledge is the practical understanding of a subject. This is an…