Would You Like The Ability To Hide Columns In The Pipeline?
Jan 17, 2007
Alot of people complain, legitamately, that they wish to remove columns from the SSIS pipeline that they know are not going to be used again. This would help to avoid the "clutter" that can exist when there are alot of columns in the pipeline.
If you are one of those people then click-through below, vote and (most importantly) add a comment. The more people that do that - the more likely we are to get this functionality in a future version.
SSIS: Hide columns in the pipeline
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=252462
Hi everybody, I've a big Matrix with 50 columns and 20 rows, I would like to hide the columns in 'real time' is it possible? for example:
-col_1 -col_2 -col_3 -col_N row_1 a b c d row_2 a b c d row_3 a b c d row_N . . . .
If we don't want to see the col_1 and col_3 , we can give a click and hiding, is it possible? maybe with the option 'visibility can be toggled by another report item' or expanded and collapsed option, some idea?
I have a matrxi report with 6 Columns (Col 1, 2, ...6). Based on the parameters I select I wanted to hide 3 of the columns (Col 2, 4 and 6. But the problem is when I hide the columns, it leaves a gap. That means, the matrix report shows Col 1, 3 and 5 with gap in between the Columns.
There is no Column Visibility property in matrix report.
Please help me as I am looking to solve this issue for a long time now.
I want detailed subtotals but I don't want the "Amount" and "Amount Class." values in the details of my Matrix.
I've not found other way to show the "amount" and "amount class." in the subtotal without adding those values in the details are of the matrix. But when I select "Hide" in the properties those values are hidden in the subtotal as well. Is there any way to hide a value in the details area but not in the subtotals ?
I'm currently investigating whether SQL Reporting Services might be the right solution to some reporting requirements I have been given (web reports).
There is a requirement that some of the columns are hidden to begin with and can be shown by clicking on the header of one of the other columns. E.g. There is a list of clients and a "Total Revenue" column, clicking on the "Total Revenue" column header would show an additional 5 or so columns which show a breakdown of different figures that make up the total revenue.
So my question is this: Is it possible to show/hide columns by clicking on the headers of other columns in the report?
The first six months will be displayed in each case.
But when i choose 6 months i just want to display first six months only.when i choose 12 months  i want to be displayed first 12 months columns only and for 18 months all the columns,.
I thought the built in expression Level() would show me be drill down level. I can't see to get that to work. I'd like to determine my drill down level so I can hide columns based on that.
The scenario is as follows: I have a source with many rows. Each row has a column called max_qty_value. I need to perform a calculation using another column called qty. This calculation is something similar to dividing qty/(ceiling) max_qty_value. Once I have that number I need to write an additional duplicate row for each value from the prior calculation performed. For example, 15/4 = 4. I need to write 4 rows to the same target table as in line information for a purchase order.
The multicast transform appears to only support fixed and/or predetermined outputs. How do I design this logic in SSIS to write out dynamic number of rows to a target table.
Hello, My problem is this. I did a dts of my old access database to a new sql 7 db. I used Access 97 as the frontened and connected the two databases using linked tables. I made sure all the indexes and such were correct in the SQL database before linking + made sure all the security was corret. When I open a table in the access frontend that is linked to the sql backend, I cannot update or change any data even though I am part of the NT administration group which is given full permissions to the database. Any reason why?
Hello, My problem is this. I did a dts of my old access database to a new sql 7 db. I used Access 97 as the frontened and connected the two databases using linked tables. I made sure all the indexes and such were correct in the SQL database before linking + made sure all the security was corret. When I open a table in the access frontend that is linked to the sql backend, I cannot update or change any data even though I am part of the NT administration group which is given full permissions to the database. Any reason why?
If I have a lot of dropdowns of similar type but only differ in one field, is there a way to set parameters so that the dropdownlist can specify what the parameter values are? For example dropdowns: expenseaccount, incomeaccount, assetaccount SqlDataSource: SelectCommand="select accountid,accountname from accounts, accounttypest where accounts.accounttypeid=acounttypes.accounttypeid and accounttypes.accounttypename=? order by accountname" <SelectParameters> <asp:Parameter Name="accountTypeName" Type="string" /> </SelectParameters> ------- I would like expenseaccount to be able to specify "EXPENSE", incomeaccount to specify "INCOME", etc... Or is sqldatasource reusage bad practice? (damn vs2k5 colors)
Hi all. We have a mix of informix and mssql server and I want to knowif something we do in informix has an analogous feature in MSSQL. Wecan define a "row type" in informix, like so:create row type name_1(fname char(20),lname char(20));The when we create any table that includes a first and last name, wedo so using this row type like so:create table sometable(name name_1, some column,...etc)This allows us to set a standard for certain common fields and avoidshaving different developers build the same type of field in more thanone way, different lengths, etc.Is there a similar function in MSSQL server?
I want to incorporate this code but I dont know how to import Microsoft.SqlServer.Dts.Pipeline in an Integration Services Project template. I was thinking of putting this code in the script task but still, I cant import Pipeline. Add reference list does not have it as well. Please let me know how to incorporate this code. Thanks!
Code: if (ComponentMetaData.RuntimeConnectionCollection["SourceFileConnection"].ConnectionManager != null) { cm = DtsConvert.ToConnectionManager(ComponentMetaData.RuntimeConnectionCollection["SourceFileConnection"].ConnectionManager);
Can some participants in this forum provide ideas on suggested training for somebody with a basic knowledge of SQL (started working in this field 6 months ago and still feel like a newbie beginner).
I assume I should focus on getting T-SQL nailed down before delving into the SSIS piece of SQL Server 2005?
Maybe there are a few companies that seem to give good practical knowledge and are located in the greater Los Angeles metro area?
Is it better to get certified through a 1 week or 2 week course via companies like Trainingcamp - does that really give you the practical skills to jump up several levels at one does at work?
Hope I can get some suggestions or direction on where I could best post this question for answers.
Hi all. We have a mix of informix and mssql server and I want to know if something we do in informix has an analogous feature in MSSQL. We can define a "row type" in informix, like so:
create row type name_1(fname char(20),lname char(20));
The when we create any table that includes a first and last name, we do so using this row type like so:
create table sometable(name name_1, some column,...etc)
This allows us to set a standard for certain common fields and avoids having different developers build the same type of field in more than one way, different lengths, etc.
I have a user that says that after she installed the ActiveX control for printing, that she can no longer export the reports. Has this happened to anyone else? What settings do I need to check on her machine to fix this problem?
I am using a FTP Connection Manager and the configuration setting for it is being set in the package configuration xml file. So the xml file contains the Ftpserver, FTp server username and password. The package is picking up the values from the xml file and is executing successfully. I have to do this because I was not able to provide an expression to the Connection Manager Server Password property.
Now, I want to pick up the ftp details from a database table and set it in the xml file during runtime. Is this possible? OR something like using the
Kindly look at the items in bold. Is this possible? Then I can set the value of the variable in the package before the FTP connection manager task is executed.
After a period of use (typically 1-2 hours), an application using SQL Server Express stops writing to the database, whilst still continuing to read from it. As far as I can tell, no error message is generated by any operation. The application is written in VB Express and is executed on several machines, with SQL Server Express on a central machine. Re-starting the application on the affected machine restores normal operation.
The problem on one machine does not cause a similar problem elsewhere at the same time (hence it appears to be an issue with the relationship between the application and SQL Server)
This does not appear to relate to the response to Q asked by sgcook on 9 may as we are not using user instances, but only a bare minimum connection string.
This is probably obvious, but how do I split a pipeline. I.e. I've got a data source with 200 columns - I need to split this into 20 pipelines each containing 10 of the original columns.
I am setting up filtered replication(tran repl) for a table that is 2+ TB in size. The filtered rows are less than 1% of total size.When I run the snapshot, sometimes it complete in 27 mins(I have already optimized the snapshot agent profile) and sometimes it runs for ever. Mostly it runs forever, so the chances of it finishing soon are really very less.I have infact created filtered index just to cater to the query in the filter i.e. on the filter predicates. The reason its flipping flopping is due to the incorrect plan that it chooses. I can force the right index and the filtered query runs just fine but. I cannot use the query hints when I am adding the filter in replication as its not allowed.
I then went ahead and created the plan guides to force the best plan. But the plan guide was not used because even if there is a single space in the actual query that replication runs in the background the plan guide will not be used(testing on plan guides has proved this). The query needs to be ditto same including whitespaces and carriage returns. I then ran profiler and found that there is a view that replication creates and executes to extract the filtered rows and BCP it. Got the text of the view and created the plan guide on it but it was still not used because sp_helptext output seems to be different than what is actually being fired by replication internally.
Further, I collected the verbose log with level 2 and tried to find out if the main query shows up there with no success.
(a) I can't use the plan guide because I do not know the exact syntax(I mean I have the query but I do not know the white spaces, carriage returns, etc) and
(b) I cannot use query hint while filtering the article.
1. I cannot afford to have downtime. 2. I cannot use backup and restore option either.
OS: Windows XP Pro, SP2 SQL: SQL Server 2005 (Server 9.0.2047) SP1 VS: Visual Studio 2005 RTM.050727-4200 --------
After upgrading our SQL Servers to SP1 we have lost the ability to debug .net CLR stored procedures. The debugging process worked fine until we installed the Service Pack. All other features we use test out fine after the upgrade. The only and very important feature that does not work is debugging CLR procs. In fact this problem is happening in every environment we have. Be it SQL2005 running on Windows XP SP2, Windows Server 2003 SP1, Window Server 2000 SP4. SQL Server remote or local on the machine doing the debugging. All 5 developer workstations with Visual Studio in virtual machines or installed on in base OS are behaving exactly the same. Every developer on our team has the exact same problem regardless of their configuration after the upgrade to the service pack 1. All developers and environments were working correctly before the upgrade.
Even building a new SQL box from scratch and installing all tools and creating a new clean database we fail with same error.
We have verified CLR Debugging is enabled for the database connection. We have dropped the connection and recreated it.
Here is an example of the output from the output window:
This is the output up to the CLR call
The thread 'sql_test2 [62]' (0x68c) has exited with code 0 (0x0). The thread 'sql_test2 [62]' (0x68c) has exited with code 0 (0x0). Auto-attach to process '[1072] [SQL] sql_test2' on machine 'sql_test2' succeeded.
Output AFTER the CLR Call
Auto-attach to process '[1072] sqlservr.exe' on machine 'sql_test2' succeeded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLBinnSqlAccess.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_32System.Data2.0.0.0__b77a5c561934e089System.Data.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_MSILSystem2.0.0.0__b77a5c561934e089System.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_32System.Transactions2.0.0.0__b77a5c561934e089System.Transactions.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_MSILSystem.Security2.0.0.0__b03f5f7f11d50a3aSystem.Security.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_MSILSystem.Xml2.0.0.0__b77a5c561934e089System.Xml.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'CIS_Release6a', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_MSILMicrosoft.VisualBasic8.0.0.0__b03f5f7f11d50a3aMicrosoft.VisualBasic.dll', No symbols loaded.
'sqlservr.exe' (Managed): Loaded 'C:WINNTassemblyGAC_MSILSystem.Configuration2.0.0.0__b03f5f7f11d50a3aSystem.Configuration.dll', No symbols loaded.
Program Locks up with blank screen and hangs. If we stop the Debugging then the correct results will appear in the output window.
<CIS><Header><MessageType>FosterParent</MessageType><Status>-1</Status><ErrorMsg><Code>-100</Code><Description>Requested Record Not Available</Description></ErrorMsg></Header></CIS>
I am trying to get this to work - but it only returns minutes & seconds:
Function Seconds2mmss(ByVal seconds As Integer) As String Dim ss As Integer = seconds Mod 60 Dim mm As Integer = (seconds - ss) / 60 Seconds2mmss = String.Format("{0:0}:{1:00}", mm, ss) End Function
Can anyone help me out? I am not that familiar with VB.
We have deployed an SSIS package successfully to production. We needed to apply SP1 to fix a different issue and now have encountered a new problem. We have numerous Data Reader Sources in different Data Flow Tasks that connect to a IBM iSeries (DB2) source. Pretty simple extracts that have worked fine in the past. They pump the data into staging tables on the SQL2K5 instance running the package (64-bit).
After we applied SP1 however, all of the Data Reader tasks fail AFTER they successfully copy the records with the following error.
[iSeries Invoice Details [1]] Error: System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.SqlServer.Dts.Pipeline.DataReaderSourceAdapter.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPrimeOutput(IDTSManagedComponentWrapper90 wrapper, Int32 outputs, Int32[] outputIDs, IDTSBuffer90[] buffers, IntPtr ppBufferWirePacket)
If I delete the source and destination and recreate identical transforms, they work fine, but I don't feel like rebuilding all of the extracts. Any ideas! The problem occurs in all environments that we've tried.
TIA, Michael Shugarman P.S. I just tried the SP2 CTP, but that doesn't fix the problem.
Hi I have created a simple SSIS project on my client that carries out 4 Data Flow tasks, each one copying a few hundred rows from an Oracle 10.0.2 database. This works OK and will also run in debug mode fine.
I have copied the package to the file system on our development server and get the following error when in debug mode:-
[DTS.Pipeline] Information: Validation phase is beginning. Progress: Validating - 0 percent complete [OLE DB Source [1]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Server.user" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed. [DTS.Pipeline] Error: component "OLE DB Source" (1) failed validation and returned error code 0xC020801C. Progress: Validating - 50 percent complete [DTS.Pipeline] Error: One or more component failed validation. Error: There were errors during task validation. Validation is completed [Connection manager "Server.user"] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80004005 Description: "Error while trying to retrieve text for error ORA-01019 ". Validation is completed
If you go to the source of each flow task and select preview you can retreive the data.
I have a situation where we get XML files sent daily that need uploading into SQL Server tables, but the source system producing these files sometimes generates duplicate records in the file. The tricky part is, that the record isn't entirely duplicated. What I mean, is that if I look for duplicates by grouping the key columns, having count(*) > 1, I find which ones are duplicates, but when I inspect the data on these duplicates, the other details in the remaining columns may differ. So our rule is: pick the first record, toss the rest of the duplicates.
Because we don't sort on any columns during the import, the first record kept of the duplicates is arbitrary. Again, we can't tell at this point which of the duplicated records is more correct. Someday down the road, we will do this research.
Now, I need to know the most efficient way to accomplish this in SSIS. If it makes it easier, I could just discard all the duplicates, since the number of them is so small.
If the source were a relational table, I could use a SQL statement to filter the records to remove the duplicates, but since the source is an XML file, I don't know how to filter these out in the pipeline, since the file has to be aggregated to search for dups.
I have an existing application that programmatically builds SSIS 2005 packages.
I'm trying to get to working with the February CTP of SQL Server 2008. Having changed all the 2005 references to 2008 references and things like IDTSComponentMetaData90 to IDTSComponentMetaData100, my application compiles okay now, but hits a problem when it tries to create a Data Flow task.
The code which worked fine before (and seems to still be the recommended way in Books Online is):
In one of my environments, I need to grant the ability to view all the logins and agent jobs to an account, but I don't want to give him "sysadmin" or "securityadmin".
Hi, My package hangs and the log says DTS.Pipeline: Validation phase is beginning. Any ideas why this is happennig? This same package runs fine when I run it without turning on the transaction.
I have several stage to star (i.e. moving data from a staging table through the key lookups into a fact table) ETL transformations in a single SSIS package. Each fact table has a different set of measures but the identical foreign key set, e.g. ConsultantKey, SubsidiaryKey, ContestKey, ContestParamKey and MonthKey.
Currently I have to replicate the key lookup (Surrogate Key Pipeline, or SKP) for each data flow. If I could cache each dimension one time in the package and reuse it for each stage to fact it would be much more efficient.
Is there a way for me to reuse a common data flow?
