Skip to main content

SSIS_Variable_Timestamp_Logging

 

Variables in SSIS

-          Purpose of a variable is to move data from one task to another or one package to another.

-          Dynamic connections

-          Loop the files /folders

Variables in SSIS:

There are 2 types of variables in SSIS

1.    System Variables

a.    Pre-Defined

b.    Read Only

c.     Stores Package Execution Related info

d.    Presented in Name Space: System

e.    Scope: Local (Task) or Global (Package)

2.    User Variables

a.    User –Defined

b.    Read and Write

c.     Presented in Name Space: User

d.    Scope: Local (Task) or Global (Package)

Syntax to Access Variables:

NameSpace::VariableName

Ex:

·         System::PackageName

·         User::FilePath

*** Variables and Name Space names are case sensitive in SSIS

TimeStamp:

@[User::FolderPath]+"\\"+"Sales_"+ GETDATE()  -- Error received as we need to convert the getdate to string.

To write this: E:\MSBI_VIMP\Test\Sales_20200817

Write like this: @[User::FolderPath]+"\\"+"Sales_"+ REPLACE( SUBSTRING( (DT_WSTR,50) GETDATE(), 1, 10),"-","")



Logging:

Customized Logging: Success

·         Here we need to create a table for log in our destination DB, see script below.

·         Create a variable RowCnt in package to get the no. of rows loaded.

·         Add Execute SQL Task-> Add sql command as

 

Insert into SSIS_Logs values (?,Getdate(),?,'Success.Package executed

successfully.')

 

·         Add parameter in Parameter mapping. Here, we are passing the parameter 0 (package name) and 1 (Row Count) in above Insert Query in question mark (?).



Script:                               

Use TableName_Stage

go


CREATE TABLE SSIS_Logs

(

            ID                                int                    primary key identity(1,1),

            PkgName                   Varchar(100) NOT NULL,

            PkgExecTime              datetime         NOT NULL,

            Row_Count                int                    NOT NULL,

            PkgExecStatus           varchar(100)  NOT NULL

)

SELECT * FROM SSIS_Logs

go


Ready Made Logging: Failure

·          Goto SSIS menu ->Logging->it will open configuration window

·         Select Provider type as -> SSIS log provider for SQL server and add

·         Select connection in configuration.

·         In details tab select the event types like on error, on task failure.

·         Sysssislog table will be created in system tables to get the details of the package failure details.

Tardis_stage DB-> System Tables -> sysssislog

 

File Existence: (Whether File Exists or Not)

1.    Check whether source folder has files or not

2.    If file exists,

a.    Load the data

b.    Move the file to Archive folder by adding time stamp

3.    If file does not exists,

a.    Log the details in table






C# code:

//Declare variable

 string x;

           

 //Assign the value to variable

  x = Dts.Variables["User::FilePath"].Value.ToString();

 

 //Check whether file exists or not

 //If exists return TRUE or FALSE

  Dts.Variables["User::FileFlag"].Value = File.Exists(x);




Popular posts from this blog

How to remove special characters in the Data(using Script component Task and Regex)

Start with the Below screen shot ( Before Script Task) and you will come to know about the senario. Note : The Script Component also works for the special characters within the Data as well . for ex "A$hish" Hover over the below Regex. it will show > Show potential fixes > in that Select System.Text.RegularExpressions. The Below will disappear.  Use the Below Script.  public override void Input0_ProcessInputRow(Input0Buffer Row) { Row.CustomerID = RemoveSpecialCharacters(Row.CustomerID); Row.NameStyle = RemoveSpecialCharacters(Row.NameStyle); Row.Title = RemoveSpecialCharacters(Row.Title); Row.FirstName = RemoveSpecialCharacters(Row.FirstName); Row.LastName = RemoveSpecialCharacters(Row.LastName); Row.CompanyName = RemoveSpecialCharacters(Row.CompanyName); Row.SalesPerson = RemoveSpecialCharacters(Row.SalesPerson); Row.EmailAddress = RemoveSpecialCharacters(Row.EmailAddress); Row.Phone = RemoveSpecialCharacters(Row.Phone); } public static string ...

DBMS

Database : database can be defined as a collection of information stored at a particular place written in a predetermined manner, belongs to a particular topic.  DBMS: (DATA BASE MANAGEMENT SYSTEM):  it is defined as software which is present inside the database which is used o maintain or manage the data within the database .                                 In the above diagram, if the user needs any info from the database. He has to contact with  dbms , then the dbms goes to the database, takes the information from the  database and gives it back to the user. Database models :  these models are broadly divided into the following types: 1.         FMS/FDMS 2.         HMS/HDMS 3.         NDBMS 4.         RDBMS 5.      ...

Different Schemas in DWH

  SCHEMAS:   Star Schema: -       Star Schema means Dimension Tables are directly linked to the Fact Table. Snow Flake Schema: -       Dimension Tables are indirectly linked to the Fact Table OR -       Dimension table is linking to another dimension table .       Star Schema   Snow Flake Schema 1 Has redundant data and hence tough to maintain/change 1 No redundancy and hence more easy to maintain/change 2 Less complex queries and hence easy to understand 2 More complex queries and hence less easy to understand 3 Less foreign keys so faster execution time of cube. 3 More foreign keys so take more execution of cube 4 Has De-Normalized Tables 4 Has Normalized Tabl...