SSIS Performance 32 Bit V 64 Bit
Jun 19, 2006
I have been working on a project last few months. I have developed the project on my laptop, which is resonably powerful. It runs through fine within 9 mins with sample data set.
If I replicate the same environment on a 64 Bit machine with 32 Bit Win 2003 and SP1, it takes just over 7 mins.
If I rerun it on a 64 Bit machine with 64 Bit Win 2003, it takes between 21 and 24 mins.
We are executing the packages via dtexec on a command prompt.
Has anyone seen same behaviour?
Thanks
Sutha
View 19 Replies
ADVERTISEMENT
Jan 5, 2007
Hi All,
Our client is a multinational Cigar company and we have been building SSIS packages for them for extraction of data into data marts.
We tried to do all the tasks using T-SQL procedures also and tried comparing the performance of procedures and SSIS packages.
We found out that procedures are better than SSIS packages in performance(Time taken). Why is this so?
We expected the SSIS packages to be much quicker but it was not the case...
Thanks,
Prakash.P
View 2 Replies
View Related
Feb 7, 2006
hi,
I have a scenerio in Which I have to handle millions of rows.
The Data should be read in Chunks and Written into Custom Destination into Chunks, How will I Acheive it.
Thanks
Dharmbir
View 1 Replies
View Related
Jan 31, 2007
I am currently in the process of migrating DTS packages to SSIS. I am finding that most of the packages are running faster, but some of them are taking longer to execute.
The DTS package copies data from our Production server to Development. It uses a Copy SQL Server Objects Task to copy only the data from about 50 tables. This takes about 3.5 minutes. I created the exact same package in SSIS using the Transfer SQL Server Objects Task and it is running about 5 minutes.
Another package I am having this problem with is only copying data from 1 table using a Copy SQL Server Objects Task. This package executes in 19 minutes. I have created the exact same package twice. Once using Transfer SQL Server Objects and once with a data flow task. The Transfer SQL Server Objects package takes about 50 minutes while the Data Flow package takes about 40 minutes.
As I said most of the packages are faster with SSIS so this is why I am confused on these couple where they are just copying the data.
Any help is much appreciated.
Thanks,
Sam
View 3 Replies
View Related
Feb 12, 2007
Hi:
I've written new SSIS packages to do what DTS packages did and the
performance I'd say is about 20 times slower. In this package, I have
a loop that loops through different servers based on server entries in
a SQL database. Each loop pumps 10 tables. The source query is set by
a variable and the destination table is set also by a variable, since
all this data goes to the same tables on the SQL server and the
definitions are all the same on the source server (Sybase). It's still going and has taken about 12 hours to pull roughly 5 million records.
The source query ends up being:
SELECT *, 'ServerName' FROM SourceTable1 WHERE Date >= Date
The 'ServerName' , the "sourcetable1" and the "Date" are all set by
variables which in turn build the source query variable.
Anyway, I just mention this for completeness--I would think setting
the variables which have anything to do with the pump's performance. How can I check
to see where the performance is getting held up?
Also, I have checked via ping the timeout to the 3 servers. The slowest one pings in about 62 ms, the fastest at 1 ms, and the other somewhere in between.
View 7 Replies
View Related
Feb 26, 2007
Hi,
I have 4 execute packages say A, B, C, D. These packges contains packages for data transfer in turn.
Package A - 15 packages
Package B - 15 packages
Package C - 20 packages
Package D - 20 packages
When i run these packages one at a time in the order A, B, C, D then the execution time is around 17 mins.
If I make a parent package and put A, B, C, D in sequence in it, executing this parent package increases the execution time to arnd 50 mins.
Pacakge A, B, C, D doesnot run in parallel. They run one at a time. So i was wondering why the there is so much time difference.
Please let me know if theres some configuration settings to make the parent package efficient.
All the test conditions are same in both the cases. The source and the target are SQL server which are on the same machine as the SSIS.
Thanks,
Vipul
View 5 Replies
View Related
Feb 7, 2008
I have a bunch of packages that take views and create tables from them. Some of the views are rather complex, but the packages themselves are very simple... drop and re-create a table using the data from a view on the same server. We create a new DB for each year, and this year we've upgraded to a new server with SQL 2005, so our DTS packages on the 2000 SQL server had to be recreated in SSIS on the new server. No problem, as I said the packages are really simple. But when I create the packages in SSIS they now take an extremely long time to execute, and I cannot figure out why.
For instance, one DTS package would take approximately 5 minutes to run when the view contained hundreds of thousands of rows and the underlying tables contained millions. But now, even with MUCH smaller tables (since it's the beginning of the year, new DB) the SSIS package I created on the new server takes over an hour, literally. The view that the SSIS package is using to create the table only takes about 15 seconds to execute in management studio (only about 16,000 rows). How can this possibly take so long??
the new server is virtually the same hardware-wise... 4 x 2400mhz, 4gb ram, win2k3 server
View 14 Replies
View Related
Aug 20, 2007
Hi All,
I'm working on a conversion project and I'm trying to compare performance of SSIS with Other ETL Tools, especially Informatica PowerCenter. Which one do you think is better ETL performer, when source and destination being SQL Server databases. Is there any benchmark available?
Thanks.
View 3 Replies
View Related
Sep 27, 2006
Hi,
our package have design like this,
OLEDBSource à Derived Column à Lookup
|
Matching Records Un Matched Records
| |
OLEDBCommand OLEDBDestination
(Update) (Insert)
and our source & destination table are oracle. when we execute the package the performance is very low and some times its showing like processing ( yellow color) even for 1 hrs .what could be the problem.can any one help us.is there any reason like when we use orcale database this will slow down the performance of package
Jegan
View 3 Replies
View Related
Nov 3, 2006
Hello again,
I'll just throw my question: how could I increase SSIS-performance?
I have a really heavy job with thousands of records my base selection, then I perform some lookups (I replaced most of them by sql) and derived columns (again, I replaced as much as possible by sql). Finally, after a slowly changing dimension task, I do update/insert on a given table. Is there a trick to speed up lookups and inserts (something like manipulating the buffer sizes - just asking).
Fact is that I replaced a script task by pure sql-joins and gained 6 of the 12 hours this job took.
Any ideas?
Greets,
Tom
View 2 Replies
View Related
Oct 5, 2007
Dear Friends,
I always use this forum to find support and to try help others.
But this time I need to receive your feedback about my package that will be in prodution in few weeks.
So.. could you give me your opinions? I prefer i write the comments in the blog, but you can write here to...
http://pedrocgd.blogspot.com/2007/10/bicasestudy-package-v2.html
Kind regards and thanks!!
Good work!!
View 1 Replies
View Related
Apr 23, 2008
. Have you faced any performance issues with SSIS?
View 6 Replies
View Related
Feb 19, 2008
I created a dataflow that transferred about 1 million records from a SQL database on one server to a differend SQL database on the same server. The processing took about 30 minutes. I used the Fast Load option.
I then created a "Execute SQL Task" and wrote a "SELECT * INTO TABLE" and this processing took about 30 - 60 seconds.
Can someone tell me why creating a Data Flow Tak would take so much longer or give differences between the two options above? Can someone give some pointers on how to make a Data Flow task more efficient?
Thanks.
View 11 Replies
View Related
Aug 28, 2007
Hello Everyone,
Can any one update me up performance turning of SSIS and what difference would it make if I change the default value of this two parameter in each Data Flow.
DefaultBuffermaxRows
DefaultBufferSize
Also update me on what is these parameters used for.
Thank you
View 3 Replies
View Related
Nov 14, 2005
I have a multi-threaded C# application that loads a bunch of tables into ado.net datasets in memory for surrogate key lookups. Depending on what else is going on, it can process 100,000 to 170,000 rows per minute and usually utilizes 20-30% of each cpu.
View 8 Replies
View Related
Mar 7, 2008
Hi,
I created procedure which completes execution in 20 mins in sql server 2005 but if i kept the same procedure in Execute SQL Task in SSIS and executing means, it is taking 3 hrs.
Is there any way to increase the performance for the above same.
Any help would be appreciated
Thanks
Dinesh
View 9 Replies
View Related
Oct 11, 2006
On 32 bit SSIS installations, both of the following performance counter objects are visible in perfmon.
SQLServer:SSIS Service
SQLServer:SSIS Pipeline
On 64 bit SSIS installations, only the following is available.
SQLServer:SSIS Service
The SQlServer:SSIS Pipeline counters are nowhere to be found.
Should I re-install? Is this a known issue with 64 bit SSIS?
P.S. Remote or local access administrative access with perfmon makes no difference, the "SQLServer:SSIS Pipeline" performance counters don't appear in the listbox when connecting to Windows 2003 x64 server.
View 4 Replies
View Related
Jul 27, 2006
Hello,
I have been running massive ssis packages and testing the performance.
This is my execution design:
I have a main package that gets a list of packages to execute from a table.
Then using a foreach loop in send the package to execute ( somewhere in the middle i delete the corresponding old log file for that package ), each of the packages configures themselves from the parent package variables.
What i have been analysing tells me that for example a package runs in 2 minutes and then the time wasted from the end of the package to the start of the other task is in average 3 to 6 minutes... thats alot... since i run about 20x12 packages witch gives me of wasted time about 20 hours.
My question is... what can be causing the delay between the end of package and the start of the other one...
The tasks types i am using in the execution controller package are:
Foreach loops, For Loop, File System task, Execute Package Task and some Execute SQL Tasks
Best Regards,
Luis Simões
View 15 Replies
View Related
Apr 1, 2008
Hi,
Can anyone tell me how to check the performance of a SSIS package?
Thanks
View 4 Replies
View Related
Nov 17, 2006
Hi,
Can anyone tell me the best way in SSIS to log performance at control flow level i.e. per task I have in my control flow and what performance characteristics it is possible to log.
Thanks in advance
View 1 Replies
View Related
Dec 28, 2007
We have SSIS installed and everything is working great. We are now to the point of wanting to tune one of our longer running packages and the Performance counters are not working. At all. They show up ok but the counter is always at 0. Is there anything special I have to do to get this to work.
One comment I fould was that the Performance Logs and Alerts service needs to be running to see these counters. I tried to start it and it immediately quit. I set it to automatic startup up and run a package. The counter still read 0.
Is there anything else out there I can try to get these counters to return somthing. Thanks.
Aaron
View 9 Replies
View Related
Apr 13, 2006
My apologies if this is a very basic question, but I am having a very difficult time finding the answer.
My very, very simple dataflow task is PAINSTAKINGLY slow. (It took over an hour to transwer @300,000 records). I'm doing no transformations whatsoever. In fact, the only reason I'm using the Data Flow component here is for its error tracking capabilities.
Here's a brief description-
1) The source is an OleDB datasource object that uses an OLEDB connection to access a SQL Server 2000 database.
2) The output from the source is dumped directly (no data transformations) into an OLEDB Destination Object (uses an OLEDB connection to access a View on a SQL Server 2005 database). Individual row errors are pushed to a seperate logging table.
Based on the advice of an article I read, I removed the "OleDB Destination" object and used the records from the OLEDB source as the input to a RowCount Transformation. This still took a SIGNIFICANT amount of time. I'm guessing that my problem is with using an OleDB Source component???? That seems really strange though... wouldn't it be optimized? What are my workaround options?
Any help is _much_ appreciated.
Thanks,
Jess
View 4 Replies
View Related
Nov 8, 2006
Hi,
I have multiple data flow tasks defined in my package. The task of the package is to extract data from Oracle/InfoLease tables and put them on to a SQL Server 2005 database.
Listed below are few queries that I had:
1. In SSIS package, I need to add "Data Conversion" component to convert from Unicode string datatype to String datatype. This was not required in SQL Server 2000 DTS package.
2. By default, Individual transformation is created for each column. Is there a way, to create one transformation for all columns.
3. This SSIS package is being executed as part of a job. The execution time takes around 33 mins.
The same functionality of the SSIS package was replicated in form of SQL Server 2000- DTS package and was executed in form of a job. This execution got completed in 9 mins.
So there has been a drastic increase in job execution time. Are there any ways to increase performance.
Any help will be highly appreciated.
Thanks.
View 5 Replies
View Related
Sep 24, 2007
Hi -
I am facing 2 problems :
PROBLEM 1 :
We have a few packages that run pretty fast on a desktop server with 2 Gig RAM, Dual processor (approx 4-5 hours). But the same packages run very very slow on the another server containing 8 CPU and 12 Gig RAM (ran for 24 hours without completing).
PROBLEM 2 :
The CPU% ranges from 40-80% and the PF usage is stagnant at 2GB on desktop server for the same package. But in the 8CPU server, the CPU % ranges from 0-10% but the PF Usage raises from 750 MB to 8 GB.
This has become critical to our application.
TIA,
Shabs
View 4 Replies
View Related
Jul 1, 2007
Dear friends,
I need your feedback, tips and opinions to improve my dataflow described in my blog.
http://pedrocgd.blogspot.com/2007/07/ssis-dataflow-performance-i.html
Thanks!!
View 8 Replies
View Related
May 16, 2007
Hi,
I would like to know if any one out there has the same performance issue with ssis packages once you move from a 2GB of RAM, 4 processors to a 16 GB of RAM with 8 processors. My SSIS packages performance is extreemly slow especially for one of my loading packages. The execution time for that package is almost double as compare to the execution time on the smaller box (2GB, 4 processor). There is no transformation for this package. It is just one lookup for existing or not and then do the insertion. This package is handling insertion of millions of rows.
The 2GB of RAM, 4 processor box is 32 bit with SP1 and the 16GB of RAM, 8 processors is 64 bit with SP2. However, I did try to force the execution of the package in 32 bit on the 64 bit server. The execution time is still the same as it is executed in 64 bit mode.
If any one out there has experienced the same thing and has known the cause to the problem, please let me know I would greatly appreciated.
Thanks,
MN
View 3 Replies
View Related
Feb 22, 2008
I'm working on a fairly straight forward data transfer package and have found that the package runs dramatically faster when I run the package inside BIDS than with DTExec. When I run the package on the server using debug in BIDS, the job completes 1 million rows in around 6 minutes. When I run DTExec with the same package on the same server it is much slower and the package takes roughly 25 minutes to complete.
I know this sounds crazy and that it's supposed to be the other way around with DTExec running much faster, but I'm stumped as to what could be causing the issue. The machine this is running on is a two processor, dual core CPU with GB of RAM and I'm using terminal server to login and create the package with BIDS on SQL Server 2005 SP2.
The main feature of this package is a Foreach container that uses an ADO record set to loop over a set of values from a control table. There are a large number of iterations so the package loops frequently, but the data flow task is fairly simple and uses an OLEDB source and OLEDB destination to transfer data between two SQL Server 2005 databases.
The package works in either BIDS and DTExec, but I'm really puzzled why it would run so much faster inside BIDS?
Thanks in advance,
-Russ
View 7 Replies
View Related
Feb 7, 2007
Hello!
I have a general question concerning the impact on the performance of massive parallel data imports in one SSIS-package.
We have a Database on a SQL2005 SP1-Server (2 Xeons 3,8 Ghz, 4GB of RAM) for a report web-app which is updated every day with data of the last year/3 years. The data is extracted from several different DBs on multiple machines at different locations. Right now, there are imports/transformations from 7 companies at 3 locations. The table has ~80 columns and about 2 Mio. Rows. I built a SSIS-Package with one companies import and added the others by c&p-ing all the tasks in the package and changing connection parameters and values. Soon there will be 6 more companies to do imports with, and there will possibly be about 20 some day.
Now, when these 7 imports run parallel, there are 3 simultaneous imports from the same Source Server. Sometimes one of these imports seem to hang up. I cannot reproduce it, when I run the package 2 or 3 times, it´s gone. So I put some of the imports in line to reduce the parallel working tasks to 4. Then the problem dissappears. The "MaxConcurrentExecutables" Value is set to 6. "Retain same connection" is set "TRUE".
My questions, regarding stability and performance, are:
1.) Is it better to do those imports in seperate packages, if yes can I schedule multiple packages to execute parallel at the "SQL Server Agent"?
2.) Or should they be combined in one package, running (partly) parallel?
3.) What is the appropriate value for the "MaxConcurrentExecutables" Value and what options do I have to speed up those imports?
View 7 Replies
View Related
Jan 9, 2007
Hi,
When I installed SP1 on my SQL Server 2005, the performance of the package designer dropped enormously. I have to wait like 30 seconds after I drag-and-drop a datasource from the toolbox to the design area. Also it takes a very long time to run the very simple package. Not to mention opening some complicated package that was done before applying SP1. The package scheduled in SQL Agent went from 3-4min to 5-6min.
Can anyone address this issue? Maybe there's something I did wrong?
While installing SP1 I ecountered strange behaviour (some messages about file locking, rebooting and stuff), but in the end it said that it was successful
Best Regards
Wojtek
View 8 Replies
View Related
Nov 15, 2007
I have a SQL Server 2005 Package (32 bit, 4 processors/16 GB of RAM, 3 GB switch for SSIS) with 1 DataFlow and 5 Custom Components:
> An OLEDB Source that reads a stoed proc from Sybase
> 3 Transform Components that perform: 6-required fields check, 4-lookups (3 cached, 1 non-cached), and a calculation that is performed(called) from a .NET Library.
> A SQL Destination that uses the .NET BulkCopy Class to insert the data.
It takes about 3 hours to load 1 million records that contain 46 columns (the largest being a varchar(100)). I have tried a good number of tuning options as well as indexing/SQL techniques to improve perfromance.
Some pieces of info:
1) We dynamically create the buffer during runtime based on an SSIS variable.
2) The actions of the components are determined at runtime through config files.
We do these two things so we can have multiple data flows run through the same package for better scalability and flexibility. Since we are dynamically creating the buffer I can NOT use any of the out of the box components because of lineage and metadata inconsistency issues.
3) I noticed on PerfMon that I am spooling buffers to disk after 100K rows.
I realize this is bad, but what are my alternatives?
Should I go to 64 bit? Is there anything else I can do?
Thanks in advance,
Anthony
View 10 Replies
View Related
Dec 18, 2007
Dear all,
Right now I built an SSIS package to transform data from external source into local database server. I schedule it to be processed at that database server (ex Server A). Is there any difference performance if I replaced the SSIS package to be processed at another server (ex Server B) ? I'd like to separate the process because I want to reduce workload in Server A by removing the SSIS process to Server B. Am I correct ?
Thanks in advance,
Hery
View 2 Replies
View Related
Feb 13, 2006
Is there any information around what the SSIS packages are doing in the first 5-10 seconds of execution, and ways to speed this process up?
View 3 Replies
View Related
Mar 24, 2006
SQL Server 2005 is installed on a brand new 64-bit server (Windows 2003 x64 std. Edition, 2.4 Ghz AMD opteron- 2cpu, 8.8 Gb of RAM). There is barely few hundred rows of data scattered among few tables in one database.
SQL server and SSIS performace grossly degrades overnight and in the morning everything is slow including the clicking of tool bar selection.It takes 3 seconds to execute a simple select statement against an empty table.
It takes15-20 seconds to execute a SSIS package that normally would take 2-3 seconds.
But once SQL Server is restarted, everything returns to normal and the performance is good all day and then the next day everything is slow again.
Thank you for your help.
View 3 Replies
View Related
