Custom Package Configuration Functionality
Nov 14, 2007
I've been working for the last few days on some custom "configuration-like" functionality that allows me to store configuration information external from the SSIS packages that need it, without relying on the built-in Configuration features of SSIS. (Of course, if someone wants to tell me how to implement Configuration encryption and a few other features while using the built-in tools, I'm more than eager to hear that too. ) I've never had the need to do this before, and although everything is going relatively smoothly, there are a few things that I'm lacking, and the main one is validation.
Currently in order for me to use my new functionality, I need to set DelayValidation = true on my data flow tasks, or else they will fail validation (and thus fail the package) unless the design-time connection information is valid. I would strongly prefer to leave DelayValidation = False so I can more easily have "deployment sanity checks" without needing to actually execute my packages.
So here is my question:
Is there a mechanism by which I can have my own logic (implemented in a Script Task currently) executed before package validation so that I can assign values to variables which are used by connection managers, so that the end product is that the package passes validation without needing to set DelayValidation = true, and fails validation if the information in the external "configuration store" is not valid for the current deployment environment?
Any tips or tricks would be very much appreciated - thanks in advance! Thanks in advance!
View 7 Replies
ADVERTISEMENT
Oct 27, 2006
Hello All,
I have searched on this forum for a similar question but couldn't find it so I apologize if this has been asked. If so, I'd greatly appreciate a link to the question. I have created a custom task and am trying to read an xml package configuration file within my custom task. To be more specific, I have added an ADO.Net Connection to my database onto the package and have generated the appropriate tags within my package's configuration xml file. I'm not seeing the classes I should use to access the configuration file of the package I've created. Does anybody have any ideas how I can accomplish this or a link to a document that might cover the material? Thanks!
Jay_G
View 3 Replies
View Related
Feb 13, 2007
Hello,
I have a report of processes which either go into a Log table or an Error log table. They also have start / finish datetimes and a timespan value. So I want to report the process elapsed time or the error condition, depending on which table the process finished up in.
I then compare the elapsed time to the timespan and then show either a red flag (errored) green flag(worked within timespan) or yellow flag(worked but ran over allotted time) depending on what happened.
Is this possible?
I think it would be easy for me to get the info I want into a dataset using .net code but how do you link that in with SSRS?
Any help appreciated,
Thanks.
View 1 Replies
View Related
May 23, 2006
Hi --I was wondering if this is a bug when I add new data in my table SSIS Confiurations and give wizard a new Configuration filter the package configuration wizard can not see the new values --the old values from the previous configuration are still showing---is there any known workaround or forced refresh I can do
thanks in advance Dave
Background:
SQL Package Configurations are most important because they provide the possibility of a central configuration store for your entire enterprise!!!!!!!! and is in my mind the only way to go
http://sqljunkies.com/WebLog/knight_reign/archive/2005/01/24/6843.aspx
Wizard results:
Name:
ETL
Type:
SQL Server
Connection name:
ETLConfiguration
Any existing configuration information for selected configuration filter will be overwritten with new configuration settings.
Configuration table name:
[dbo].[SSIS Configurations]
Configuration filter:
PT_CUST_ABR
Target Property:
Package.Variables[User::gsPreLoad].Properties[Value]
Package.Variables[User::gsPostLoad].Properties[Value]
Package.Variables[User::gsLoad].Properties[Value]
Package.Variables[User::gsFlatFilename].Properties[Value]
Package.Variables[User::gsFileName].Properties[Value]
Package.Variables[User::gsCDOMailTo].Properties[Value]
Package.Variables[User::gsCDOMailSubject].Properties[Value]
Package.Variables[User::giRecordCount].Properties[Value]
Package.Variables[User::giFileSize].Properties[Value]
Package.Variables[User::giBatchID].Properties[Value]
Package.Variables[User::gdFileDateCreated].Properties[Value]
Package.Connections[MyDatabase].Properties[ServerName]
Package.Connections[MyDatabase].Properties[InitialCatalog]
USE [ETLConfiguration]
GO
/****** Object: Table [dbo].[SSIS Configurations] Script Date: 05/23/2006 13:34:35 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[SSIS Configurations](
[ConfigurationFilter] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValue] [nvarchar](255) COLLATE Latin1_General_CI_AS NULL,
[PackagePath] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValueType] [nvarchar](20) COLLATE Latin1_General_CI_AS NOT NULL
) ON [PRIMARY]
View 3 Replies
View Related
Feb 5, 2008
Hi all,
I have finished the development of an SSIS package but I would like to make it a bit more flexible by adding a configuration file. The script migrates a database from a source to a destination and syncs with another source.
So there is the source db and the destination db. However, I have noticed that standard SSIS configurations can get a little tricky. For instance, I have two connection managers for the destination db (one that points to the schema and one that doesnt). So my question is: Is there any way to create a config file thta you can input a few attributes like sourceDB, destinationDB, schemaName and get the package itself to parse and process it?
Thanks,
Yannis
View 1 Replies
View Related
Feb 2, 2007
Hi,
I am planning to develop a single package that will download files from ftp server, move the files to internal file server and upload it in the database. But I want to run this package for multiple ftp file providers. For each provider the ftp server might be different and the transformation to upload the files into a database table might be different.
So can I create a single package and then multiple configuration files (xml), which will contain the details fo the ftp file providers and then pass the xml file as a parameter while executing the package. The reason being that the timings of fetching the files is different for each ftp file provider and hence cannot be combined into one.
Is this possible?
Thanks for your help.
$wapnil
View 6 Replies
View Related
Sep 10, 2007
Deployed Report having SSIS package as source do not work when Indirect Package configuration is used in ETL package. It seems ETL package when called/executed from Report manager does not recognize environment variable to pick up the dtsconfig file.
The Report works when Direct package configuration is used to same dtsconfig file.
What could be the reason? Any solution for this? This will cause our build/deployment to QA and Prod very difficult.
View 1 Replies
View Related
Sep 14, 2006
Hi guys,
Here's the deal. I have a child package, (say, pack01.dtsx), which uses a dtsconfig file for its connection string, which can be called from other packages, but which also can be called by itself.
However I also have another package (say, pack02.dtsx) which uses the same dtsconfig file for its connection string. It calls on pack01.dtsx.
When I use DTEXECUI and run pack01.dtsx, specifying the proper .dtsconfig file, it goes well. But when I try and run pack02.dtsx, an error occurs saying pack01.dtsx connection cannot be established.
How do I pass the connectionstring being used by pack02 to pack01, without having to remove the configuration file setting of pack01? Can a Parent Package configuration and a configuration file try and map to the same property?
Hope someone could help. Thanks.
View 7 Replies
View Related
Mar 7, 2006
Hi all,
my problem is that: I do read informations (e.g. connectionString) from configuration file by one custom assembly.
My environment is RS2005, VS.NET 2005 Prof.Edition, WIN2003 Server SP1, .NET Framework 2.0.50727
The scenario is that:
A text box into my report require a function contained into MyCustom.dll. This code do read information from MyCustom.dll.config.
This file contains:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<connectionStrings>
<add name="IS2009cfg" connectionString="data source=servernameinstancename;initial catalog=dbname;user id=xx;password=yy;" />
</connectionStrings>
<appSettings>
<add key="DefaultHeaderLogo" value="This is a test" />
</appSettings>
</configuration>
Using
ConnectionStringSettings conn = ConfigurationManager.ConnectionStrings["IS2009cfg"];
the result is null.
Also using
string defaultHeaderLogo = ConfigurationManager.AppSettings["DefaultHeaderLogo"];
the result is null.
MyCustom.dll and MyCustom.dll.config are located into locations of Report Server and Report Designer, with this Build Events on succesfully build:
rem Copy custom assembly to ReportDesigner location...
copy "...mydeveloplocation...inDebugMyCustom.dll" "C:Program FilesMicrosoft Visual Studio 8Common7IDEPrivateAssembliesMyCustom.dll"
copy "...mydeveloplocation...inDebugMyCustom.dll.config" "C:Program FilesMicrosoft Visual Studio 8Common7IDEPrivateAssembliesMyCustom.dll.config"
rem Copy custom assembly to ReportServer location...
copy "...mydeveloplocation...inDebugMyCustom.dll" "C:Program FilesMicrosoft SQL ServerMSSQL.4Reporting ServicesReportServerinMyCustom.dll"
copy "...mydeveloplocation...inDebugMyCustom.dll.config" "C:Program FilesMicrosoft SQL ServerMSSQL.4Reporting ServicesReportServerinMyCustom.dll.config"
The question is simple: where I do locate configuration file for obtain value during report execution?
Many thank's in advance
Claudio Barca
Turin - Italy
View 2 Replies
View Related
Jan 14, 2008
Hi,
I am using custom dll in script component in SSIS package. This dll is looking for some configuration settings and dsplays the message as "Configuration section could not be found in the configuration source" . Please tell me the configuration source it looks for.
View 3 Replies
View Related
Mar 6, 2006
Hi Guys,
Here's the current scenario:
I am creating a windows application to manage my ssis packages.
My SSIS packages use package configurations, and by default connect to the "SSIS Configurations" table in sql server.
However, I want it to connect to a view instead of this table, so i create a view "vwSSIS_Configurations" and point the package configuration to use the view in sql server.
The application SHOULD be capable of altering the package configuration's configuration and switch its connection between the default "ssis configurations" table, and the view.
Is this possible? I cannot think of a way which I can have my application switch the package configuration's connection.
Thanks,
kervy
View 3 Replies
View Related
Aug 8, 2007
I have created a package configuration to read the connectionstring for the oledb connection manager.
Saved the file as Environment.dtsConfig in the same directory as the other packages.
Do the packages refer to the configuration file automatically?
Does it make a difference if I use the dtexec to start the ssis packages since there is now a configuration file?
Thanks
View 4 Replies
View Related
Mar 29, 2007
I would like to set package configuration.But dont know how to.
What i want to achieve is that i want to change connections based on test ,development and production environment of the same package...
How can i do that without me changing the connections manually to each task based on environment..
using package configuration i dont know how to...Please let me know.
View 17 Replies
View Related
Apr 5, 2006
Hello,
I'm developing some packages in SSIS and I need to implement a configuration file to load the server's connections and other parameters, so I can use this same connection for all packages.
Have anyone a step by step to perform this configuration ?
I exported a Connection Manager to a .dtsConfig file, but I don't know how to use this config file.
Any help is welcome !
Thanks,
Guber
View 1 Replies
View Related
Oct 3, 2006
Here is the scenario we are
trying to use; (it a bit long winded please bare with me)
There are 2 Packages Parent
and Child. Both have 2 Connection Managers (€śConfiguration€? & €śData Connection€?).
The €śData Connection€? has an invalid connection set to start with this is so
that we can deploy this to multiple places and use a SQL table to hold the
configuration.
At runtime €śConfiguration€? is
set to a valid SQL2005 database connection via the command line or in BI but €śData
Connection€? is left invalid.
The Package Configuration point
to an SQL server using €śConfiguration€? this has the connection string for the €śData
Connection€? Connection Manager (a valid connection string). The package errors
with can not acquire connection.
The first task is to put the
Connection string of the Connection manager €śConfiguration€? in to a variable.
Then the child package is called.
The Child package has the same
2 connections €śConfiguration€? & €śData Connection€? both are Invalid at this
time.
1) We use the Package
Configuration to populate a variable from the parent package which holds the €śConfiguration€?
connection string.
2) We use a Package
Configuration to populate then Configuration€? connection from the parent
package which holds the €śConfiguration€? connection.
3) We use a Package
Configuration to populate the €śData Connection€? Connection Manager using the €śConfiguration€?
connection. Which should be Set via step 2 here.
The First task on the child
package has a bit of script to check all the connection manager return a valid
connection to make sure database are up and running before connection. This
fails at the moment return in an error.
We want to use the same code
in multiple environments and as such don€™t want to make code changes that are really
only environment changes.
IE connection string to
servers. I can€™t seem to see anything saying we are doping anything wrong but if
the connections are all set to a valid connection before running this seems to
work. When invalid connection are used this seems to fall over.
View 1 Replies
View Related
Aug 8, 2007
I have created a package configuration to read the connectionstring for the oledb connection manager.
Saved the file as Environment.dtsConfig in the same directory as the other packages.
Do the packages refer to the configuration file automatically?
Does it make a difference if I use the dtexec to start the ssis packages since there is now a configuration file?
Thanks
View 1 Replies
View Related
Aug 10, 2007
Does it matter if the configuration is by xml,sql server,parent package variable,environment variable etc.
I use xml configuration which i kind of feel easy as i can change the connection directly by typing.Not used with others much..
Is there any security issue here .
Please let me know
View 5 Replies
View Related
Sep 14, 2006
Hi,
i am haveing package which is devloped in BIDS and i am executing the package from DotNet (Windows Application C#).during development my package is pointed to Server1,Database1.when we move the package to another test environment for a different server and database (Server2,Database2). we are giving the connection string details in dtsConfig file .but Package is not takeing the new connection string instead its takeing the old connection string (design time connection) and throwing error.what could be the problem
Invoking Code (C#)
--------------------
btn_SaExecute.Enabled = false;
string pkgLocation;
Package pkg;
Microsoft.SqlServer.Dts.Runtime.Application app;
DTSExecResult pkgResults;
pkgLocation = "D:JeganSSIS PrjsInvokeTestInvokeTestData1.dtsx";
app = new Microsoft.SqlServer.Dts.Runtime.Application();
pkg = app.LoadPackage(pkgLocation, null);
pkg.EnableConfigurations = true;
pkgResults = pkg.Execute();
Data.dtsConfig
----------------------
<?xml version="1.0"?><DTSConfiguration><DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="PSLSDCJegan.thangadurai" GeneratedFromPackageName="Data1" GeneratedFromPackageID="{30ACE86F-D0D3-4575-AD46-2E9C2F34D46B}" GeneratedDate="9/14/2006 12:54:01 PM"/>
</DTSConfigurationHeading>
<Configuration ConfiguredType="Property" Path="Package.Connections[XXXX.YYYY].Properties[ConnectionString]" ValueType="String">
<ConfiguredValue>Data Source=XXX;User ID=XXX;Password=XXX;Provider=MSDAORA.1;Persist Security Info=True;
</ConfiguredValue>
</Configuration>
</DTSConfiguration>
Thanks
Jegan
View 1 Replies
View Related
Sep 1, 2006
Hi all,
Hopefully a simple question. In my c# application i can call ImportConfigurationFile(<str>) to import an xml configuration file. How do i go about importing configuration settings from a SQL configuration package? I have created this but see no way to load it. I'd prefer to use the config file created in the SQL database.
Any help would be most appreciated.
Many thanks,
Grant
View 1 Replies
View Related
Oct 17, 2006
Hi all of us,
I'm looking any .dtsconfig file as sample for my first package configuration. I've got ssis packages which might be updated on development and then moved to production. First of all, I'd like dynamically change the connections but I don't know exactly how to do that. I'm taking a look on the wizard but when I choose a Sql Server request me a table.
Any sample or link for that would be welcomed!
TIA
View 2 Replies
View Related
Aug 25, 2006
Type:
Configuration File
New configuration file will be created.
File name:
C:Test.dtsconfig
Properties:
Package.Connections[Flat File Connection Manager].Properties[ConnectionString]
------------------------------------
So lets say, I modify the connectionstring for the Flat File connection manager by editing the xml file. Next time, I run the package, it does not grab the new value. Am I misunderstanding the concept of Package Configuration?
View 3 Replies
View Related
Mar 14, 2007
Hi All,
I have couple of properties in my package Configuration
(1) Lets me know the Folder path - where I have file processing
(2) Connection String
When working in the Development machine, I am able to play around these values and test the application. I have tried both XML Configuration and SQL Server Configuration
But during deployment in the Test Server, the values in the Configuration are not getting reflected automatically and takes the values assigned during deployment
I tried importing the XML file and it works. But the values are not persistant.After running the package, the value gets resetted to the old value. Every time I run my package, I am reassingning my package variables.
Be it XML Configuration File or SQL Server, I would like to have the variables pulled out from the Configuration I am setting during the deployment. the values should be persistant and should refer the updated values.
Any help will be highly appreciated...thanks
View 10 Replies
View Related
Dec 7, 2006
hi everyone
I am putting my connection string in package configuration file, which i create through package configuration option.
But connection is not picking up value where as other varibles are picking up values from same file.
Is there is any other setting that we have to set for connection string.
Thanks and regards
Rahul Kumar
View 3 Replies
View Related
Apr 12, 2007
Hi, I'm facing a problem that I'm trying to solve. I report it in the following hoping somone can help me!
the context:
- I have a solution containing almost 20 packages
- I have a main package containing the sequences (Sequence Container) calling all others packages (Execute Package Task)
- every package has the connections to its own DBsource ; so the package X has the its own connections to DBsourceA and DBsourceB, the package Y has the its own connections to DBsourceB, DBsourceC, DBsourceD and the destination DB is unique for all the packages.
My problem is:
I have the necessity to define in the best way a config (one for all the dtsx, possibly) that allow me to manage easily the switch from the developing env to the deployment env, basically for the DB connections.
Which is the best way to do this and can anyone tell me the steps to follow?
View 8 Replies
View Related
Mar 6, 2006
I've been searching for an answer to my question quite some time now and I've not been able to figure it out yet.
Situation:
- I've created a SSIS package containing a bulk insert task.
- I've added a package configuration containing the appropriate connection manager (i.e. dev, beta or live)
- CreateDeploymentUtility = true
- I've copied the deployment folder to our beta server and I started the manifest file to install the package to the sql 2005 server, after that I specified the config file location and changed the value so the approriate connection manager is used.
- When I execute the package from the sql server the package doesn't read the value from the xml config file, it uses the connection which was originally specified in the package, whereas when I run the package from my BIDS it is reading the value from the xml config file?
I can't seem to figure out why this is happening? am I missing something here?
Thnx.
View 1 Replies
View Related
Jan 3, 2007
I'm facing a little problem concerning package configuration that is using a SQL table.
I have 4 exactly the same Db's on the same SQL server, each with another name of course (Dbipa, DbIpB, DbIpC & DbIpD).
My developers created 4 exactly the same SSIS packages with 4 exactly the same jobs, each for his own DB only the connection settings are different.
Now I was trying to manage this with only one SSIS package by using an SSIS Package configuration table in each DB.
I thought it would be possible to start the package with the following parameter
/SET Package.Connections[SQL].InitialCatalog;DBIPA OR
/SET Package.Connections[SQL].InitialCatalog;DBIPB OR
/SET Package.Connections[SQL].InitialCatalog;DBIPC OR
/SET Package.Connections[SQL].InitialCatalog;DBIPD
But this doesn't work because before he changes the initial catalog in the connection named SQL, he already did the loading of the SSIS_Configuration table. So he point always to the same config table.
Is there a way to change the connection before the package configuration is being executed.
Ludo Bernaerts
View 1 Replies
View Related
Nov 9, 2006
1) We are doing data migration from SQL Server 2000 OLTP design to SQL Server 2005 OLAP design.
2) We have used SSIS packages and data flow tasks in which we mentioned connection strings for source and target containers.
3) We have a master execute package which contains series of execute packages in relational order.
4) When we execute this master package, we should be able to dynamically specify different source and connection strings for all packages.
5) In master execute package, we generate an XML configuration file using SSIS -> Package Configurations... using Connection Managers -> Particular connection
6) Now, we change connection string to point to another database after adding this new OLE DB connection in Connection Manager of each and every package.
7) When we save, build and execute master package it is still executing migration packages against the old database. Please let me know what I am missing.
View 3 Replies
View Related
Jul 17, 2006
Hi,
Programmatically, how will i be able to know if a package is using package configurations?
Thanks.
View 10 Replies
View Related
Jul 2, 2007
I have a package that executes succesfully before package is configured.
but when package is configured to use configuration details either from SQL Server configuration table or XML configuration file, gives an error :
[Connection manager "XYZ"] Error: An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
Can anyone help me out with this ?
Experts !!! this is my second post, my previous post is still unanswered.
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1793362&SiteID=1
View 7 Replies
View Related
May 29, 2007
Hi,
I am new to SSIS. I have an SSIS package which we are storing it in the application server ( not in SQL Server ). We are storing the database name etc.. in package configuration. Where should I store the configuration file? In app server or in the database server. Also please tell how to execute the package stored as file system.
Thanks
ramesh
View 11 Replies
View Related
Apr 13, 2006
Am I missing something, I can't find any way to specific the XML configuration file when I try and run my package in Debug of BIDS. My OLE DB connections are all failing due to a lack of a password. I can set the password in BIDS but its forgotten as soon as I start Debug or build the solution. The only way I seem to be able to run my package (now that I have enabled configurations) is to de-select the enable configurations box.
I've tried putting a configuartion file in the Deployment folder, the bin folder and the project root folder all to the same result.
View 2 Replies
View Related
Jul 29, 2006
When I try the option to store package configuration into a SQL Server table, it keeps asking me for a filter and I dont' see any options on the drop down list. How do I set this up? I know how to set the package configuration in an XML file and that worked pretty OK for me. But there is now a need to store the package configuration in a SQL Server table now. Please help me with this problem.
View 1 Replies
View Related
Oct 16, 2007
I have a package that executes succesfully before package is configured.
but when package is configured to use configuration details either from SQL Server configuration table or XML configuration file, gives an error :
Error: An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
I am using Hit Software OLEDB provider for DB2/400 Connection
Can anyone help me out with this ?
View 10 Replies
View Related
