Skip to main content

SSIS_Containers_Checkpoint_Security_Utility

 Containers in SSIS:

There are 3 containers in SSIS

1.   Sequence Containers (Folder)

a.    It is similar to folder

b.    It groups the control flow tasks and then Sequence them

Parallel Execution will happen if you don’t put arrow




Sequential Execution will happen if you put arrow:

 

2.   ForLoop container:

a.    If Condition is known at design time

b.    It is similar to loops in any language.

c.     It will run/repeat the task for FIXED no of times.




Ex: Insert 10 numbers in a table.



--SQL code:

declare @x int

--Initialization

set @x=1

while (@x<=10)

            Begin

                        --Action

                        insert into tbl_numbs values (@x)

                       

                        -- Incr/Decr

                        Set @x=@x+1

            End

go

3.   ForEachLoop Containers:

·         If Condition is unknown at the design time.

·         It is similar to loop in any language.

·         When we do not know ‘How many times we need to run the loop’

·         Loop is executed for each Object/File etc

Ex: Get the data from All flat files (.txt) and load into SQL table

 

Build and deploy the package:

DEV Tasks:

1.    Create a SSIS Package/Perform Unit Testing

2.    Implement Logging in SSIS

a.    SSIS Menu->Logging- SQL and windows event

b.    Event Handlers (Customize Logging)

3.    Build and Check-in the code to VSTF/TFS

a.    Ispac file needs to be uploaded

b.    Path of Ispac file: <drive>:\<path>\bin\Development

c.     Provide Deployment Guide

DBA Tasks

1.    Download all files from VSTF to DWH server.

2.    Create Integration Services Catalogs in SQL Server of DWH

3.    Deploy the package using ispac file

a.    Deploy to SQL Server/Deploy to file System

Note: Implement Configurations for connection strings

4.    Scheduled a package Using SQL Server Jobs

 

Check Points in SSIS

·         Check Points restart the package from the point of failure.

·         When we configure Check Point in a package:

                                          i.    Creates the XML/text file.

                                         ii.    Takes one of the following two actions:

1.    Saves the file if the package failed/stopped

2.    Deletes the file if the entire package runs successfully.

·         SSIS will reference the checkpoint file, when we re-run the packages during failures.

Note: Check Point cannot start the package at row level. It can start at task level.

 

Check Point configurations:

·         Configure check points at Control flow

                                          i.    Right click on control flow designer->Properties

                                         ii.    Check Points:

1.    Never: A checkpoint file is not used

2.    Always: Select this to always restart the package from the checkpoint

3.    IfExists: Select this to restart the package only if the check point file is available.

·         Configure the following at Task Level:

                                          i.    FailPackageOnFailure=TRUE

 


Security to SSIS:

Package Security:

·         Use ProtectionLevel property to secure SSIS package.

·         SSIS encrypts sensitive information, such as a password or a connection string with a password

·         ProtectionLevel property had following options:

                                          i.    EncryptSensitiveWithUserKey(default)

                                         ii.    DontSaveSensitive

                                       iii.    EncryptSensitiveWithPassword(recommended)

                                       iv.    EncryptAllWithPassword

                                         v.    EncryptAllwithUserKey- Its same as password

                                       vi.    ServerStorage

 

SSIS Utilities:

·         DTEXEC: It is used to execute a SSIS package from command prompt or from a SQL Server.

·         DTUTIL:

                                          i.    The utility can copy, move, delete, or verify the existence of a package.

                                         ii.    Cannot be deployed to SSIS catalog

                                       iii.    Can be deployed to msdb

Call SSIS package Using Command Prompt:

1)   File system:

a.    dtexec.exe /f “D:\Demo1.dtsx”

2)   SQL server:

a.    Dtexec.exec/ sq Demo1/ser kunal

Note: Do not use extension.dtsx when the package in SQL server

About XP_cmdshell:

·         Xp_cmdshell is extended stored procedure.

·         Dtexec.exe is available at C:\programfile\Microsoft SQLServer\100\DTS\Binn\dtexec.exe

·         /f- means file system Deployment Package

·         /sq – means SQL Server Deployment Package

SSIS Deployment options:

There are 3 ways to deploy a SSIS package

1)    Using .ispac file/Manifest file

2)    Using Command Line Executable called DTUTIL.exe

3)    Using SSDT-> Right click on Project->Deploy

Popular posts from this blog

How to remove special characters in the Data(using Script component Task and Regex)

Start with the Below screen shot ( Before Script Task) and you will come to know about the senario. Note : The Script Component also works for the special characters within the Data as well . for ex "A$hish" Hover over the below Regex. it will show > Show potential fixes > in that Select System.Text.RegularExpressions. The Below will disappear.  Use the Below Script.  public override void Input0_ProcessInputRow(Input0Buffer Row) { Row.CustomerID = RemoveSpecialCharacters(Row.CustomerID); Row.NameStyle = RemoveSpecialCharacters(Row.NameStyle); Row.Title = RemoveSpecialCharacters(Row.Title); Row.FirstName = RemoveSpecialCharacters(Row.FirstName); Row.LastName = RemoveSpecialCharacters(Row.LastName); Row.CompanyName = RemoveSpecialCharacters(Row.CompanyName); Row.SalesPerson = RemoveSpecialCharacters(Row.SalesPerson); Row.EmailAddress = RemoveSpecialCharacters(Row.EmailAddress); Row.Phone = RemoveSpecialCharacters(Row.Phone); } public static string ...

DBMS

Database : database can be defined as a collection of information stored at a particular place written in a predetermined manner, belongs to a particular topic.  DBMS: (DATA BASE MANAGEMENT SYSTEM):  it is defined as software which is present inside the database which is used o maintain or manage the data within the database .                                 In the above diagram, if the user needs any info from the database. He has to contact with  dbms , then the dbms goes to the database, takes the information from the  database and gives it back to the user. Database models :  these models are broadly divided into the following types: 1.         FMS/FDMS 2.         HMS/HDMS 3.         NDBMS 4.         RDBMS 5.      ...

Different Schemas in DWH

  SCHEMAS:   Star Schema: -       Star Schema means Dimension Tables are directly linked to the Fact Table. Snow Flake Schema: -       Dimension Tables are indirectly linked to the Fact Table OR -       Dimension table is linking to another dimension table .       Star Schema   Snow Flake Schema 1 Has redundant data and hence tough to maintain/change 1 No redundancy and hence more easy to maintain/change 2 Less complex queries and hence easy to understand 2 More complex queries and hence less easy to understand 3 Less foreign keys so faster execution time of cube. 3 More foreign keys so take more execution of cube 4 Has De-Normalized Tables 4 Has Normalized Tabl...