Compressing Multiple Rows With Null Values To One Row With Out Null Values After A Pivot Transform
Jan 25, 2008
I have a pivot transform that pivots a batch type. After the pivot, each batch type has its own row with null values for the other batch types that were pivoted. I want to group two fields and max() the remaining batch types so that the multiple rows are displayed on one row. I tried using the aggregate transform, but since the batch type field is a string, the max() function fails in the package. Is there another transform or can I use the aggragate transform another way so that the max() will work on a string?
This should be a simple solution, but it has been a long time since I've done any query writing (mostly in Oracle) and I am stumped, so here goes:
We are in the process of converting Access database to MSSQL with web form front ends.
I have a table, all columns are nullable, and want users to be able to query from a form, which has a field for each column and defaults to a % wild card for the entered value.
I want the users to be able to put any string in any field, and have it return each row that matches that, including rows with null values in the other columns, but not the column with the entered criteria.
Here is a sample of the data:
Code: SQL> select * from test;
COL1 COL2 COL3 COL4 ----- ----- ----- ----- this is a test this is not test this is not this is test too is test too is too is too
7 rows selected.
Now, if I have this SQL run, it will return only rows that have no nulls in any columns:
Code: select col1, col2, col3, col4from test where col1 like'th%' and col2 like '%' and col3 like '%' and col4 like '%';
COL1 COL2 COL3 COL4 ----- ----- ----- ----- this is a test this is not test this is test too
Now, if I use an OR clause for each column, this mostly works, but the trouble is it will also return rows with null values for the field that has criteria entered in it:
Code: select col1, col2, col3, col4from test where (col1 like'th%' OR col1 is null) and (col2 like '%' OR col2 is null) and (col3 like '%' OR col3 is null) and (col4 like '%' OR col4 is null); COL1 COL2 COL3 COL4 ----- ----- ----- ----- this is a test this is not test this is not this is test too is test too is too is too
The idea is to only select the first 4 rows in the above example.
I was playing with ISNULL in the select clause, but all it does is substitute a string for a null, and I think CASE will do the same thing.
Is there a way I can write this query so it will return rows with NULL values in any column, except the one(column) that has user entered criteria in it?
I cannot in the life of me understand what goes wrong here... In a webapplication (C#.NET) I traced an inability to retrieve existing records to SQL Server, where I cannot do the same either. The problem is that in the parameterized query, some fields can be null, and thus when the corresponding fields in the database record are null also, they should be selected. But this won't happen for some reason.
I wrote a test SQL statement that gives me the same bogus:
DECLARE @institution int, @collection int, @serialnr int, @subnr nvarchar(50)SET @institution = 1 SET @collection = 1SET @serialnr = 240 SET @subnr = NULLSELECT ID, Institution, Collection, SerialNumber, SubNumber, AccessionYear, Sagsnummer, DanekraeNr, TaxonIdentified, Stratigraphy, TypeStatus, PlacementRoom, PlacementCabinet, PlacementDrawer, UnitNotesFROM SpecimensWHERE (Institution = @institution) AND (Collection = @collection) AND (SerialNumber = @serialnr) AND (SubNumber = @subnr) Now there is at least one row with corresponding fields values (1, 1, 240, null), but it won't be selected! What is wrong!?
Here is the content of the tbIntersect table: Car100_ID Car200_ID Car300_ID Car400_ID Car500_ID Car600_ID Car700_ID Prod_ID ID 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 19 1 3 1 1 1 1 1 20
I need to return the rows that have null data, ex: second row because Prod_ID is NULL and third row because Car300_ID is NULL. In fact I need the data from the other joint tables that correspond to these ID fields.
I have a DTSX package which reads values from a fixed-length text file using a data reader and writes some of the column values from the file to an Oracle table. We have used this DTSX several times without incident but recently the process started inserting NULL values for some of the columns when there was a valid value in the source file. If we extract some of the rows from the source file into a smaller file (i.e 10 rows which incorrectly returned NULLs) and run them through the same package they write the correct values to the table, but running the complete file again results in the NULL values error. As well, if we rerun the same file multiple times the incidence of NULL values varies slightly and does not always seem to impact the same rows. I tried outputting data to a log file to see if I can determine what happens and no error messages are returned but it seems to be the case that the NULL values occur after pulling in the data via a Data Reader. Has anyone seen anything like this before or does anyone have a suggestion on how to try and get some additional debugging information around this error?
I have a pivot transform that it believe is configured correctly but is not distributing the values accross the columns on the same row. for example.
input:
id seqno codevalue
1 A red
1 B red
2 C blue
2 A green
2 B violet
3 A green
desired output:
id Seq_A Seq_B Seq_C
1 red red null
2 green violet blue
3 green null null
what I am getting:
id Seq_A Seq_B Seq_C
1 red null null
1 null red null
2 green null null
2 null violet null
2 null null blue
3 green null null
I do have the pivot usage for the id column set to 1. I have the pivot usage for seqno column set to 2 and codevalue column set to 3. I have the source column for each of the output columns set to the lineageID of the apprpriate input columns. I have the pivotKey values set for each of the destination columns. A for column Seq_A, B for column Seq_B, C for column Seq_C. All four columns have sortkey positions set; 1 for id, 2 for Seq_A, 3 for Seq_B and 4 for column SEQ_C.
It seems like the id column's pivot usage is not set to 1 like it should but when I check it is 1.
I also have several other pivot transforms in the same data flow and they are working as expected.
I have a suspicion that there is some hidden meta data that is messed up that is over-ridding my settings (just my guess) I have deleted this transform and re-done it several times, checking each configuration value, but still getting the same result.
I have SQL Server 2012 SSIS. I have Excel source and OLE DB Destination.I have problem with importing CustomerSales column.CustomerSales values like 1000.00,2000.10,3000.30,NotAvailable.So I have decimal values and nvarchar mixed in on Excel column. This is requirement for solution.However SSIS reads only numeric values correctly and nvarchar values are set as Null. Why?
CREATE TABLE [dbo].[Import_CustomerSales]( Â [CustomerId] [nvarchar](50) NULL, Â [CustomeName] [nvarchar](50) NULL, Â [CustomerSales] [nvarchar](50) NULL ) ON [PRIMARY]
can somebody explain me how I can assign a NULL value to a datetime type field in the script transformation editor in a data flow task. In the script hereunder, Row.Datum1_IsNull is true, but still Row.OutputDatum1 will be assigned a value '0001-01-01' which generates an error (not a valid datetime). All alternatives known to me (CDate("") or Convert.ToDateTime("") or Convert.ToDateTime(System.DBNull.Value)) were not successful. Leaving out the ELSE clause generates following error: Error: Year, Month, and Day parameters describe an un-representable DateTime.
My query "select blah, blah, rank from tablewithscores" will return results that can legitimately hold nulls in the rank column. I want to order on the rank column, but those nulls should appear at the bottom of the list
I know this is probably a flick of a switch but I cannot figure out which switch. Setup is SQL Server / Stored Procedures / DAL / BLL(skipped for testing) / PL. The stored procedure queries from only one table and two columns are ignored because they are being phased out. I can run the stored procedure and preview the data in the DAL but when I create a page with an ODS linked to the DAL and a GridView I get this error. I checked every column that does not allow nulls and they all have values. I checked unique columns (ID is the only unique and is Identity=Yes in the table definition). I checked foreign-key columns for values that are not in the foreign table and there are none. Any ideas why do I get this? Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
Hi, I am getting the above error when trying to load a report into my Web Application, I have tracked the error down to one specific field in my database. Even though this field is a NVarChar field and is of size 30 it would seem that there is an issue returning the value from the field. I can write it into the database no problems but when I try to get it out of the database it returns the above error. e.g MOB 401.908.804 - Fails 0401.907.324 - okay 8239 9082 (pager) - fails Anyone got an idea on how to fix this???? Regards.. Peter.
I have an SSIS package that imports data from an Excel file, replaces any value in Excel that reads "NULL" to "", then writes the data to a couple of databases.
What I have discovered today, is I have two columns of dates, an admit date and discharge date column, and what I need to do is anywhere I have a null value in the discharge date column, I have to replace it with the value in the admit date column.Â
I have searched around online and tried a few things using the Replace funtion in Derived columns but no dice so far.Â
I am looking to create a constraint on a table that allows multiplenulls but all non-nulls must be unique.I found the following scripthttp://www.windowsitpro.com/Files/0.../Listing_01.txtthat works fine, but the following lineCREATE UNIQUE CLUSTERED INDEX idx1 ON v_multinulls(a)appears to use indexed views. I have run this on a version of SQLStandard edition and this line works fine. I was of the understandingthat you could only create indexed views on SQL Enterprise Edition?
Hi, I ve a table, I want to fetch certain rows based on the value of a Column. That column is nullable, and contains NULL values.I used the following query,SELECT Col_A FROM TABLE1 WHERE SOME_ID = 1317 AND Col_B NOT IN (8,9) Here, Col_B contains NULL values too. I need to fetch all rows where Col_B is not 8 or 9.Now, if I use "NOT IN", it does not work. I tried reading on it and got to know why it does not work. Even "NOT EXISTS" does not help. But still I've to fetch my values. How do I do that?Thanks & Regards,Jahanzeb
I have tried doing a search, as I figured this would be a common problem, but I wasn't able to find anything. I know that my SP is functional because when I use VWD execute the query outside of the webpage, I get the correct results -however I have to ensure that a field is either entered, or set to <NULL>. In my SET's I want it to use the wildcards. What I want is to do a search (plenty of existing topics on that, however none were of help to me). If a field is entered, then it is included in the search. Otherwise it should be ignored. In my VB I have the standard stored procedure call, passing in values to all of the parameters in the stored proc below: CREATE PROCEDURE dbo.SearchDog@tagnum int,@ownername varchar(50), @mailaddress varchar(50),@address2 varchar(50),@city varchar(50),@telephone varchar(50),@doggender varchar(50),@dogbreed varchar(50),@dogage varchar(50),@dogcolour varchar(50),@dogname varchar(50),@applicationdate varchar(50)AS IF @tagnum=-1 SET @tagnum=NULL SET @ownername = '%'+@ownername+'%' SET @mailaddress = '%'+@mailaddress+'%' SET @address2='%'+@address2+'%' SET @city = '%'+@city+'%' SET @telephone='%'+@telephone+'%' SET @dogcolour='%'+@dogcolour+'%' SET @dogbreed='%'+@dogbreed+'%' SET @dogage='%'+@dogage+'%' SET @doggender='%'+@doggender+'%' SET @dogname='%'+@dogname+'%' SET @applicationdate='%'+@applicationdate+'%' SELECT DISTINCT * FROM DogRegistry WHERE ( TagNum = @tagnum OR OwnerName LIKE @ownername OR MailAddress LIKE @mailaddress OR Address2 LIKE @address2 OR City LIKE @city OR Telephone LIKE @telephone OR DogGender LIKE @doggender OR DogBreed LIKE @dogbreed OR DogAge LIKE @dogage OR DogColour LIKE @dogcolour OR DogName LIKE @dogname OR ApplicationDate LIKE @applicationdate ) AND TagNum > 0GO I don't know why it is creating links inside my SP -ignore them. TagNum is the primary key, if that makes a difference. On the webpage, it ONLY works when every field has been filled (and then it will only return 1 row, as it should, given the data entered). Debugging has shown that when nothing is entered it passes "". Any ideas?
I am trying to retrieve data from two different tables. One of the tables has more than 20 columns some of which are null. I would like to retrieve data from both tables excluding the columns which have null values. How do I do this?
Would this take care of null values in either a.asset or b.asset?
SELECT convert(decimal(15,1),(sum(isnull(a.asset,0))/1000.0)+(sum(isnull(b.asset,0))/1000.0)) as total_assets
What's throwing me off is that there are multiple a.asset or b.asset for each unique ID. It seems to work, but I'm not following the logic too well. If I were doing this in another language, I would loop through, summing a.asset and b.asset wherever it's not null for each unique ID.
How can I use "Derived Column" to check if a Datetime value is null or not and if null to insert 00/00/00 instead. ?
The background being that while using a "Derived Column" to change a Column from a (DT_DATE) to a (DT_DBTIMESTAMP) everytime I get a null value it see's it as a error.
And the column in particular has ~ 37 K blank / null fields so Im getting a lot of errors
So far I have tried to use something like
ISNULL([Column 34])
Or
SELECT ISNULL(ID, '00/00/0000') FROM [Column 34]
Or
SELECT ISNULL(au_id, '00/00/0000 00:00') AS ssn FROM [Column 34
but none seems to work [Column 34] being the offending column.
What a normally use is just a simple "(DT_DBTIMESTAMP)[Column 34]" in the expression column, which seems to work well, but here I get alot of errors
I set up a new SQL database file, in that file I allowed nulls, When I went through code to save the record, the exception is saying it doesnt allow nulls.
Before I get to involved with SQL, is it a bad practice to use nulls?
If it is what do you enter in place of the null value,
hi ive got a inert sub where i grab values from text boxes etxthe values are passed to a stored procedure however , one of these fields is a date field , but the field is not required ...so on this line if the date text box is left blank i get an error , not a valid date .Parameters.Add("@actiondate", SqlDbType.DateTime).Value = txtActionDate.Texti have tried ( the actiondate field can take nulls ..)if txtActionDate="" then .Parameters.Add("@actiondate", SqlDbType.DateTime).Value = nothing else.Parameters.Add("@actiondate", SqlDbType.DateTime).Value = txtActionDate.Textend if but this doesnt workwhat is the best way of allowing blank values to be passed to the stored procedure( it doesnt fall over with normal text / varchar fields ) thanks
I have a sproce that accepts null for one of its parameters I can execute the sproce and enter null and it works fine, it returns all rows. When I try doing this with my GridView and the SQLDataSource it does not work. I need some help in understanding how the SQLDatasource wants a null. Here is what the parameter row of the SQLDataSource looks like. <asp:ControlParameter ControlID="EnteredByText" DefaultValue="Null" Name="EnteredBy" PropertyName="Text" Type="String" ConvertEmptyStringToNull="true" /> In my sproce I have setup the parameter as follows; @EnteredBy Nvarchar(50)=Null In my WHERE Clause I have: WHERE (tblClient.EnteredBy = @EnteredBy OR @EnteredBy IS NULL)
I wonder if someone out there can help me. I am writing an ASP application to query a MSSQL database. The users will be able to use one or all of 4 columns. There may be time when the columns are empty (null). How can I write a select query to ignore null values? A rough example of what I am talking about it below. select * from table where value1='something' value2=<null> value3='something' value4=<null> I would like to ignore the null values so that in effect the statement would just do the following. select * from table where value1='something' and value3='something' I realize my syntax is wrong but I think you get the idea. Any thoughts?
I have a stored procedure that allows users to select addresses based on partially supplied information from the user. My procedure seems to work fine in all but a few cases. If a user decides to select all the rows for a particular country the procedure below does not return any rows even if the rows exist. I tracked this down to the fact that for Non US countries I set both the StateCode and Country Code to nulls. . Below is a partial version of my code. Can anyone show me how I can do a "Like" Search even if some of the fields in the row contain null values? Thanks 1 CREATE PROCEDURE dbo.Addresses_Like( @SendTo VarChar(50) = Null 2 , @AddressLine1 VarChar(50) = Null 3 , @AddressLine2 VarChar(50) = Null 4 , @City VarChar(50) = Null 5 , @StateCode VarChar(2) = Null 6 , @ZipCode VarChar(10) = Null 7 , @CountryCode VarChar(2) = Null) 8 9 10 Declare @SearchSendTo VarChar(50) 11 Declare @SearchAddressLine1 VarChar(50) 12 Declare @SearchAddressLine2 VarChar(50) 13 Declare @SearchCity VarChar(50) 14 Declare @SearchStateCode VarChar(2) 15 Declare @SearchZipCode VarChar(10) 16 Declare @SearchCountryCode VarChar(2) 17 18 If (@SendTo Is Null) 19 Set @SearchSendTo = "" 20 Else 21 Set @SearchSendTo = @SendTo 22 23 If (@AddressLine1 Is Null) 24 Set @SearchAddressLine1 = "" 25 Else 26 Set @SearchAddressLine1 = @AddressLine1 27 28 If (@AddressLine2 Is Null) 29 Set @SearchAddressLine2 = "" 30 Else 31 Set @SearchAddressLine2 = @AddressLine2 32 33 If (@City Is Null) 34 Set @SearchCity = "" 35 Else 36 Set @SearchCity = @City 37 38 If (@StateCode Is Null) 39 Set @SearchStateCode = "" 40 Else 41 Set @SearchStateCode = @StateCode 42 43 If (@ZipCode Is Null) 44 Set @SearchZipCode = "" 45 Else 46 Set @SearchZipCode = @ZipCode 47 48 If (@CountryCode Is Null) 49 Set @SearchCountryCode = "" 50 Else 51 Set @SearchCountryCode = @CountryCode 52 53 54 Select AddressID 55 , SendTo 56 , AddressLine1 57 , AddressLine2 58 , City 59 , StateCode 60 , ZipCode 61 , CountryCode 62 , RowVersion 63 , LastChangedDateTime 64 , OperID 65 From Addresses 66 Where SendTo Like RTrim(LTrim(@SearchSendTo)) + "%" 67 And AddressLine1 Like RTrim(LTrim(@SearchAddressLine1)) + "%" 68 And AddressLine2 Like RTrim(LTrim(@SearchAddressLine2)) + "%" 69 And City Like RTrim(LTrim(@SearchCity)) + "%" 70 And StateCode Like RTrim(LTrim(@SearchStateCode)) + "%" 71 And ZipCode Like RTrim(LTrim(@SearchZipCode)) + "%" 72 And CountryCode Like RTrim(LTrim(@SearchCountryCode)) + "%" 73 Order By CountryCode, City, AddressLine1, AddressLine2, SendTo
I want to write a query to bring out any records in a table where any of its fields contain a null value. I could do it in this way:
select *
from customers where name is null or role is null or email is null u003c/p>u003cp styleu003d"margin:0cm 0cm 0pt">u003cfont faceu003d"Times New Roman" sizeu003d"3">u003cfont coloru003d"#000000">However I want to perform a similar function for a number of tables and the method above would mean more sql and maintenance.u003cspan>Â u003c/span>I was hoping there was a simple way to do the same thingu003cspan> ie find any rows with a blank fieldu003c/span>?u003cspan>u003cfont faceu003d"Arial" sizeu003d"2">Â u003c/font>u003c/span>u003c/font>u003c/font>u003c/p>u003c/span>u003c/font>u003c/div>",0] ); D(["ce"]);
//--> However I want to perform a similar function for a number of tables and the method above would mean more sql and maintenance. I was hoping there was a simple way to do the same thing ie find any rows with a blank field?
I have a simple table on my webpage which shows data from my database. It all worked fine apart from one part. I wanted the select statement to select only NULL values from a column, as these are classed as open queries. Once I have closed the query a 0 or 1 will automatically be posted and will no longer have a NULL value. I have a simple select statement (which I will post on here when I log in with my work computer) and at the end I have a WHERE Column = NULL. I have also tried WHERE column <> 0.0 AND column <>1.0 and that did not work. If I only did WHERE column <> 1.0, i only get the 0.0 results and none of the NULL results, but if I have no WHERE statement I get all the results including the NULL values.
Hi I have a table in my database and one of the columns is called 'Month'. Here only one month is inserted according to what the user selects from a dro down list. WHat i want to be able to do is find the average of the column 'ScoreOut' on a month by month basis. My code works fine if th database contains at least one row with every month of the year represented. I run into problems when one of the months is not represented. here is my code:string month = "February"; SqlConnection myConn = new SqlConnection("Data Source=.\SQLEXPRESS;AttachDbFilename=|DataDirectory|\Database.mdf;Integrated Security=True;User Instance=True");String selectCommand = "Select AVG(CAST(ScoreOut as float)) from tblRound where Name = '" + lblName.Text + "' AND Month = '" + month + "'"; SqlCommand myCmd = new SqlCommand(selectCommand, myConn);SqlDataAdapter da = new SqlDataAdapter(myCmd);DataTable dt = new DataTable();
The field can have null values. But I am not able to insert null values into the database. i tried using subdate="". But it inserts some min value of my calendar popup.
Is there a way that i could insert null values into smalldatetime field of the SQl database.
When I have values for the text string, everything runs smoothly. However, when the database originally had a <NULL> or when I want to delete the currently existing value and replace it with a <NULL>, I get an error message stating... "Input string was not in a correct format".
Back in the old days, when I was using Random Access databases, you just inserted a Char(0). But that doesn't work any more.
Could somebody help me. I know there is a magic bullet, I just don't know what it is.
For some reason my Stored Procs are not recognizing NULL values within my database:For example:Select *From ResultsWhere(home_Phone IS NULL)All the home_Phone vaules that are NULL are not being picked up by the query. Any ideas are appriciated. Thanks in advance everyone. RB
