Write data from Multiple tables to single flat file

It's simple if you are transferring your single table data to a single file means its one-to-one mapping. Many threads I observed at MSDN forum where user want different set of data (Header, Detail, Trailer) in a single file or data from the multiple tables to single flat file. It's not difficult in SSIS but it is tricky.

Solution:

USE 3 dataflow tasks one for each table with precedence constraints to have them execute in the correct order.

Important: 

Each destination would use its own connection manager but they would all point to the same flat file. The only thing you would need to do is in the flat file destination property grid (or advanced editor) make sure you set the overwrite flag to false for the second and third dataflows so that the file will be appended to. 

DFT 1:
OLEDB Src (header table) --> FlatFile Dest

(using flat file conn mgr 1 (filename = OutputFile.csv), overwrite set to true)

DFT 2:
OLEDB Src (details table) --> FlatFile Dest

(Using flat file conn mgr 2 (filename = OutputFile.csv), overwrite set to false)

DFT 3:
OLEDB Src (trailer table) --> FlatFile Dest

(Using flat file conn mgr 3 (filename = OutputFile.csv), overwrite set to false)

----------------------------------------------------------------------------------------------------
Get the sample package..

Package.dtsx

For step-by-step explanation, please check the following document.

PPT:   OneOutputFile.ppsx

OR

Doc:  OneOutputFile.doc

----------------------------------------------------------------------------------------------------


There are lots of other ways to do this. but in the above method, you don't need any specific programming skill. Do let me know if you have any doubt..

Convert Number to IP address

Convert Number to IP address.

Use the following query to convert the number into an IP Address..


DECLARE @IP_NUM AS INTEGER
SET @IP_NUM=1135863234 

SELECT CAST(@IP_NUM/16777216 AS VARCHAR(3)) +'.'+  
CAST((@IP_NUM%16777216)/65536 AS VARCHAR(3)) +'.'+ 
CAST( ((@IP_NUM%16777216)%65536)/256 AS VARCHAR(3))   +'.'+ 
CAST(((@IP_NUM%16777216)%65536)%256 AS VARCHAR(3))

Result:
67.179.229.194

Please give some suggestion to make it better.

Load CUBE data to SQL table using Script task

Recently I faced an issue while working with SSAS 2008 cube. I was using a data flow task and inside that I have OLEDB source (The Analysis Services 9.0 OLE DB Provider (msolap90.dll)) for fetching data using MDX query. I was running this package in BIDS, its working fine. But as I was trying to run it with DTEXEC command, it’s throwing following error:


• ERROR message:

SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E05. An OLE DB record is available. Source: "Microsoft OLE DB Provider for Analysis Services 2008." Hresult: 0x00000001 Description: "Error Code = 0x80040E05, External Code = 0x00000000:.".


Component "OLE DB Source" (2297) failed the pre-execute phase and returned error code 0xC0202009.

Environment:
SSIS package on SQL Server 2005. BIDS 2005
SSAS cube on SQL server 2008.


Solution:
I have tried many possible solutions but no success. Finally I wrote a script task code which is connecting to CUBE and dumping the cube data in flat file with comma delimited format. Later on I used this flat file to laod data in SQL table.

I would like to share this code, may this helps other.

Important: Please add the "Microsoft.AnalysisServices.AdomdClient.dll" dll to your reference.


Microsoft.AnalysisServices.AdomdClient.dll is available under the link below,http://www.microsoft.com/downloads/en/details.aspx?FamilyId=228DE03F-3B5A-428A-923F-58A033D316E1&displaylang=en
Download "Microsoft ADOMD.NET " from the link above and after running msi file you will get
Microsoft.AnalysisServices.AdomdClient.dll under folder.
C:\Program Files\Microsoft SQL Server\100\SDK\Assemblies\Microsoft.AnalysisServices.

**********************************************************
Public Sub Main()
Dim serverName As String = "Manish"
Dim databaseName As String = "CRM"
Dim databaseID As String = databaseName
Dim cn As String = "Provider=MSOLAP;Data Source=" & serverName & ";Initial Catalog=" & databaseName
Dim connex As New AdomdConnection(cn)

connex.Open()

Dim commandText As String

commandText = "SELECT [Country].[Country Code].MEMBERS ON 0, [Time].[Gregorian Calendar].[Year] ON 1 FROM [CUBE2008];"

Dim cmd As AdomdCommand = New AdomdCommand(commandText, connex)
Dim dr As AdomdDataReader = cmd.ExecuteReader(CommandBehavior.CloseConnection)

' output the rows in the DataReader
Using writer As StreamWriter = New StreamWriter("c:\myfile.txt")
While dr.Read()
For i As Integer = 0 To dr.FieldCount - 1
If (dr(i) Is Nothing) Then
writer.Write(" , ")
Continue For
Else
writer.Write(dr(i).ToString() + " , ")
End If
Next
writer.WriteLine()
End While
End Using
dr.Close()

Dts.TaskResult = Dts.Results.Success

End Sub

***************************************************************

Let me know your valuable comment to make this post better.

Processing Options for SSAS Objects

PROCESSING OPTION SSAS2008

Process Default: (Applicable for **All objects)
Performs the minimum number of tasks required to fully initialize the object. The server converts this option to one of the other options based on the object state.

Process Full (Applicable for All objects)
Drops the object stores and rebuilds the object. Metadata changes, such as adding a new attribute to a dimension, require Process Full.

Process Update (Only for Dimensions) 
Applies member inserts, deletes, and updates without invalidating the affected cubes.

Process Add (Only for Dimension & Partition)
Adds only new data.

Process Data (Dimension, cube, measure group, partition)

Loads the object with data without building indexes and aggregations.

Process Index(Dimension, cube, measure group, partition)
Retains data and builds only indexes and aggregations.

Unprocess (All Objects) 
Deletes the object data or the data in the containing objects.

Process Structure ( Only CUBE)
Deletes the partition data and applies Process Default to the cube dimensions.

------------------------------------------------------------

**SSAS objects list

• Database
• Dimension
• Cube
• Measure group
• Partition
• Mining structure
• Mining model

Script task recompilation Error

A package that has a script task runs fine in visual studio, but the package fails as a job with following error on DEV Server.

"Script could not be recompiled or run: Attempted to read or write protected memory. This is often an indication that other memory is corrupt"

OR

“Error 1 Validation error. Script Task : The task is configured to pre-compile the script, but binary code is not found. Please visit the IDE in Script Task Editor by clicking Design Script button to cause binary code to be generated.”

Answer:

• Check the pre-complied property of script task.
• Change it to TRUE
• Add the following code in your script, before imports statement.

                    Option Strict Off

                    Option Explicit On
                    Imports System

                    .

                    . 

Checking for the file?

Question:

Many times I observed on MSDN SSIS forum that developers are facing problem for checking the file in a particular folder. There are 2 types of checking:

1> Is any file exist in the folder?

OR

Count number of files in a directory through SSIS

2> A particular file is exist [for example c:\Manish\demo.txt]

Answer:

1> Is any file exist in the folder? OR Count number of files in a directory through SSIS?

For checking the file existence in a directory, there are 2 ways

1: File Property task

File Properties Task .  It will detect file existence. Check the link.

2: Script task

Step1: Take a ForEachLoop container.

Step2: Map to the particular directory.

Step3: Inside that you have to use a script task and there you can write following code to count the no of files.

Step4: Define the following variable @ package level 

User::Count

---------------------------
|   ForEachLoop             |
---------------------------
|
|            --------------
|            |Script Task |
|            --------------
|
---------------------------

Step5:

Public Sub Main()

    Dts.Variables("User::Count").Value = Dts.Variables("User::Count").Value + 1

    MsgBox(Dts.Variables("User::Count").Value.ToString())

    Dts.TaskResult = ScriptResults.Success

End Sub

    
Now you can access the count variable

2> A particular file is exist [for example c:\Manish\demo.txt]

You can use a script task to check if the file exists or not. Suppose I have demo.txt in C:\Manish then I will use a script task with code as:

Public Class ScriptMain

    Dim flag As Boolean

    Public Sub Main()

        flag = File.Exists("C:\MyFolder\demo.txt")

        If flag = True Then

            System.Windows.Forms.MessageBox.Show("File Exists")

        End If

    Dts.TaskResult = Dts.Results.Success

    End Sub

End Clas

Hope this will help you. Give your suggestion to improve this post.

Divide Single column to multiple columns

Question:
I have a column as : act=TUNING; svid=629; VCN=180; TI=949290689; TO=949291183; TT=494; FPID=(586653033-494)
Now i want to take the values of svid,VCN,TI,TO,TT,FPID into seperate columns.
How can i do it in ssis.
The column sizes are not standard all the time.
In one of my table in the DB , i have the token strings like token1 => svid= and token2 => ;
So now how can i use these tokens to pull the data of 629. 

Please check the above question @msdn forum

Answer:

Please write the following expression in your derived column (C2 is column name)
(click on the image for larger view)

After this you just need to replace the following with the blank string
act =
svid =
TT=
TI=
TO=

Do let me know if you need more help on it.