Question About Converting Column Datatypes In Order To Save Space
Nov 6, 2007
All,
I've converted datatypes in the past to something more appropriate in order to save space, and speed up the database, to great success. I'm in the middle of another such change, this one table in particular I am converting from [text] columns to varchar(max)'s, and varchar(1)'s to char(1)'s. I've made a duplicate of the table in question (which has about 25 text columns, and 20 varchar(1) columns), ported over the data, and ran 'sp_spaceused' and got the following results:
-- new table
name | rows | reserved | data | index_size | unused
Response | 39920 | 15384 KB | 15168 KB | 208 KB | 8 KB
-- old table
name | rows | reserved | data | index_size | unused
Response | 39920 | 124128 KB | 123696 KB | 200 KB | 232 KB
I didn't expect results quite this dramatic. Now to my question: Did I miss something measuring the difference? Is there something else, another method I should employ to determine the size differentials?
If this is right, it's outstanding, just unexpected. I would have been happy losing 10 or 20 MB.
I appreciate any insight anyone might be able to provide.
Two datatype problems: (1) Datetime conversion (2) Foreign Currency format.
The first problem is that I am trying to convert source data from varchar to datetime. The source data is in CSV format and is displayed as follows - '1111999052349' to represent 1-Nov-1999 05:23:49. Have converted to a numeric value and then tried to convert to datetime but this just returns the following message 'Server: Msg 8115, Level 16, State 2, Line 3 Arithmetic overflow error converting expression to data type datetime.' However you can convert to a timestamp but the value returned is not meaningful (e.g. 0x0E0000013DFA4EE8).
Problem 2 concerns the French currency. The source data is in CSV format and is displayed as follows - '00000001,9700' to represent Fr1.97. Need to convert this to a numeric field or alternatively get SQL 7 to recognise it as a money field.
New to developing. We are working on a project using asp.net and c# where data is pulled out of an sql database via multiple tables and displayed via a details view. The user can then update the information and submit it back to the database.
The problem we are encountering is:
Within the database there are multiple tables joined by a FK for example there is a Service Group ID column with the Records main table linked to the Service Group table which holds an ID column and a Service Group column so ID1 = Service Group A. Any records within the main table which has an ID of 1, is shows as Service Group A in the details view.
What we wold like to do is when a user updates the Service Group via a drop down, the ID goes into the main table rather than the Service Group name so if they select Service Group B, an ID of 2 goes into the Records main table. We aren't sure how to go about doing the conversion of Service Group B to an ID 2? We have been looking at converting datatypes but don't think it is right for what we are trying to do,
Any suggestions would be gratefully recieved Apologies if this is in the wrong forum.
how do you convert from a date to an int ? as well as converting from Varchar to and Int
in SQL server 2000 ?
I am retrieving the GetDate() which i store in column as Varchar, i then want to use it within my select statement to calulate data which i want to return i.e
DECLARE @DateValue AS VARCHAR(20)
SELECT @DateValue = ApplicationSettingValue FROM ApplicationSettings WHERE ApplicationSettingKey = 'ProcessDate'
SELECT ccy_code, NULL, xrate_date, sterling_xrate FROM SylvanTrans.dbo.SIADHP_XRate_Hist WHERE xrate_date >= (CONVERT(int, @DateValue) - 35 ) ORDER BY xrate_date
now my as you can see in my WHERE CLAUSE i want to calculate what is returned using the GETDATE() stored in my @DateValue Variable, but the conversion throws a syntax error:
Server: Msg 245, Level 16, State 1, Line 3 Syntax error converting the varchar value 'GetDate()' to a column of data type int.
what am idoing wrong? i know it is something simple, unless there is no conversion from varchar to int?, i also tried setting the datatype for my variable as datetime but i got the same sort of error with DATETIME replacing the VARCHAR in the error!!!
Firstly, i need to work out why I cannot change my datatypes(please see query)
Code Snippet
SELECT * FROM (
SELECT top 10
ref,
RecordDate,
TransactionID,
StatusChangedTimeStamp,
TransactionStatus,
PartyTransactionStatus,
BadDeliveryReason,
TradingDaysRef
FROM (
SELECT 1 seq,
'ref' ref,
'RecordDate' RecordDate,
'TransactionID' TransactionID,
'TransactionStatus' TransactionStatus,
'StatusChangedTimeStamp' StatusChangedTimeStamp,
'PartyTransactionStatus' PartyTransactionStatus,
'BadDeliveryReason' BadDeliveryReason,
'TradingDaysRef' TradingDaysRef
UNION ALL
SELECT 2 seq,
cast(ref as bigint),
RecordDate,
TransactionID,
StatusChangedTimeStamp,
TransactionStatus,
PartyTransactionStatus,
BadDeliveryReason,
TradingDaysRef
FROM dbo.ParticipantTradeStatusChange
) x
order by seq, RecordDate
) y
The error returned is:
Server: Msg 8114, Level 16, State 5, Line 1 Error converting data type varchar to bigint.
The reason for me doing this, is exporting both column names & data to a xls file.
Secondly, once i get the query to complete...I kindly ask, how would i make this query a little swifter in which, i mean, select the top 100 from the table then SELECT the columns...when i do
I thought i fixed this problem, as it WORKED about two days ago, but now I'm getting errors.
I have a series of text files, some are delimited, some are fixed width. I previously was able to import these files thru a dts package by creating the table in a stored procedure. Exple: CREATE TABLE [Pol_Dump073104] ( [Product_Type] varchar (12) NULL, [Benefit] float NULL, [Base_Premium] varchar (12) NULL, [Rider_Premium] float NULL, [Contract_Value] float NULL, )
I then import the text file into the table and then cast the float datatypes as money in a select into statement to do aggregate functions on the money fields. this worked previously, but now I'm receiving this: 'Error converting datatype char to money.' I tried to convert to float and received the same error, only with float where the word money was.
I am using bcp to import data into a table usign format file. when I put 0 in the table column(as it is shown here) it means that I am not bcp in the field values into the table. what I get in return is (NULL) for all those columns that has 0 in the format file..... My Questions is Am I saving space in the table when I use this process, or (NULL) will take space like an actual value ?
I hope I explained my question clearly... Thanks for your input
In my data archiving process , I would end up deleting hunders records from the production databases but would that help me save some DISK space immediately??? Should I run some DBCC command to get some disk space ?
if SO ..!! What should I do after deleting the records..????
I want to konw how many rows a data page could contains.
So i do some test to prove it.
The follow sql scripts i used:
create table table1
(
col1 char(2)
)
go
declare @a int
set @a=0
while @a<700
begin
insert into table1 values('aa')
set @a=@a+1
end
I add 700 recrods to table1,then use dbcc page command oversee the data page.
The result indicate that:the whole recrods are all in one data page,and the "m_freeCnt" flag equal 396. (8192-96-700*9-700*2=396)So i think i can add more records to the data page.To my surprise, after i add a new record, sql server 2005 allocates a new data page and assign the recrod to the new space....
Hello -- I'm building an app that will allow users to create their own photo galleries. At this point, I'm planning on storing all photos as byte arrays in SQL server image fields. Besides the organizational benefit, is there a space benefit to doing this? That is, if I have 1MB of .jpg's, will those same images take up less than 1MB of file space within the database? One of the reasons I ask is that most hosting plans out there seem to offer more "normal" disk space than is allocated for the database, so I'm trying to make a best plan to accommodate what will probably end up being the biggest disk space consumer in my app (the photos, that is). Any other recommendations re: this scenario (hosting, best practices) are appreciated. TIA, Eric
I am facing the problem with datatype conversions, the scenario is using derived column transformation for add additional columns and then later i am trying to impliment Slowly changing dimensions(SCM) in my job, while mapping the columns at SCM from Input to Output it gave the error like suppose if i am using the numeric(3,0) at SRC system then i converted it into single byte unsigned at derived column and it recognized at SCD but the job fail while run the package it gave the error as task can not able to conversted given data type to target system dtata type.. ifi am not given the single byte unsigned data type at dervied column at the level of SCD mapping the input to output its not accapring this mapping and return the error as can be convert from system.decimal to system.byte..
Hi, We got a problem. supposing we have a table like this:
CREATE TABLE a ( aId int IDENTITY(1,1) NOT NULL, aName string2 NOT NULL ) go ALTER TABLE a ADD CONSTRAINT PK_a PRIMARY KEY CLUSTERED (aId) go
insert into a values ('bank of abcde'); insert into a values ('bank of abcde'); ... ... (20 times)
select top 5 * from a order by aName Result is: 6Bank of abcde 5Bank of abcde 4Bank of abcde 3Bank of abcde 2Bank of abcde
select top 10 * from a order by aName Result is: 11Bank of abcde 10Bank of abcde 9Bank of abcde 8Bank of abcde 7Bank of abcde 6Bank of abcde 5Bank of abcde 4Bank of abcde 3Bank of abcde 2Bank of abcde
According to this result, user see the first 5 records with id 6, 5, 4, 3, 2 in page 1, but when he tries to view page 2, he still see the records with id 6, 5, 4, 3, 2. This is not correct for users. :eek:
Of course we can add order by aid also, but there are tons of sqls like this, we can't update our application in one shot.
So I ask for your advice here, is there any settings can tell the db use default sort order when the order by column value are the same? Or is there any other solution to resolve this problem in one shot?
Hi, We got a problem. supposing we have a table like this:
CREATE TABLE a ( aId int IDENTITY(1,1) NOT NULL, aName string2 NOT NULL ) go ALTER TABLE a ADD CONSTRAINT PK_a PRIMARY KEY CLUSTERED (aId) go
insert into a values ('bank of abcde'); insert into a values ('bank of abcde'); ... ... (20 times)
select top 5 * from a order by aName Result is: 6 Bank of abcde 5 Bank of abcde 4 Bank of abcde 3 Bank of abcde 2 Bank of abcde
select top 10 * from a order by aName Result is: 11 Bank of abcde 10 Bank of abcde 9 Bank of abcde 8 Bank of abcde 7 Bank of abcde 6 Bank of abcde 5 Bank of abcde 4 Bank of abcde 3 Bank of abcde 2 Bank of abcde
According to this result, user see the first 5 records with id 6, 5, 4, 3, 2 in page 1, but when he tries to view page 2, he still see the records with id 6, 5, 4, 3, 2. This is not correct for users. Of course we can add order by aid also, but there are tons of sqls like this, we can't update our application in one shot. So I ask for your advice here, is there any settings can tell the db use default sort order when the order by column value are the same? Or is there any other solution to resolve this problem in one shot?
Is it possible to save column mapping definitions from a Transform Data Task? The practical use is I have four tables with very similar layouts of which 200 or so columns are identical. I have various front and back office applications that require local copies of this data in various formats. It is EXTREMELY tedious to remap all of the columns for each Transform Data Task required on these applications.
Is there a way to store all of the column mapping def's and import them into a new transform data task?
I am trying to do one scheduling website for my company. Its contains tasks thats we scheduling for ourself each one and the assigning task by the boss to everyone. I want to do this with two tables. I need to save the task and assigned task in one column named as "Tasks" and to gave after the task name assigned or myself scheduled. Also after deletion of each assigned and scheduled it must save in the table with some name like deleted or any symbol. How can I do this?please help me to solve this issue.
Hi experts, I have a textbox and a upload file function in my asp.net page.User can either copy/paste their resume in text or upload their resume file and submit the application.The uploaded file will be saved into a BLOB column, but do you know if text in textbox can be saved into BLOB column? I received error message on the code:'save Applicant resume to a BLOB-image datatype column objComd.Parameters.Add(New SqlParameter("@AppResume", SqlDbType.NText)) error:Exception has been thrown by the target of an invocation.Operand type clash:ntext is incompatible with image
helo.. I want to save file it size about 200MB to the database and after that
I want to get it from the database, the colmun type that i want to
save the file to it is NVARCHAR(Max).
I am using MS sql server 2005.
I was using this code to do this with Image Or Ntext column type but
it dose not working with NVARCHAR(Max) column?
Function SaveFileToDB(ByVal FileName As String, ByVal TblName As String, ByVal FldName As String, ByVal ColumnIDName As String, ByVal RowID As String) As Boolean
Try
Dim addEmp As SqlClient.SqlCommand = New SqlClient.SqlCommand("SELECT @Pointer = TEXTPTR(" & FldName & ") FROM " & TblName & " WHERE " & ColumnIDName & " = '" & RowID & "'", MyConnection)
Dim trParm As SqlClient.SqlParameter = addEmp.Parameters.Add("@Pointer", SqlDbType.Binary, 16)
trParm.Direction = ParameterDirection.Output
addEmp.ExecuteNonQuery()
'''''''''''''''''''''''''''''''''''''
Dim bufferLen As Integer = 1048576
Dim appendToPhoto As SqlClient.SqlCommand = New SqlClient.SqlCommand("UPDATETEXT " & TblName & "." & FldName & " @Pointer @Offset 0 @Bytes", MyConnection)
Dim ptrParm As SqlClient.SqlParameter = appendToPhoto.Parameters.Add("@Pointer", SqlDbType.Binary, 16)
ptrParm.Value = trParm.Value
Dim photoParm As SqlClient.SqlParameter = appendToPhoto.Parameters.Add("@Bytes", SqlDbType.Image, bufferLen)
Dim offsetParm As SqlClient.SqlParameter = appendToPhoto.Parameters.Add("@Offset", SqlDbType.Int)
offsetParm.Value = 0
Dim fs As IO.FileStream = New IO.FileStream(FileName, IO.FileMode.Open, IO.FileAccess.Read)
Dim br As IO.BinaryReader = New IO.BinaryReader(fs)
Dim buffer() As Byte = br.ReadBytes(bufferLen)
Dim offset_ctr As Long = 0
Do While buffer.Length > 0
photoParm.Value = buffer
appendToPhoto.ExecuteNonQuery()
offset_ctr += bufferLen
offsetParm.Value = offset_ctr
buffer = br.ReadBytes(bufferLen)
My.Application.DoEvents()
Loop
br.Close()
fs.Close()
Return True
Catch ex As Exception
MyErrStr = ex.Message
Return False
End Try
End Function
Public Function ReadFileFromDB(ByVal MyCommandText As String, ByVal FileColumnNumber As Integer, ByVal DSTFileName As String) As Boolean
Try
Dim command As SqlClient.SqlCommand = New SqlClient.SqlCommand(MyCommandText, MyConnection)
Dim stream As IO.FileStream
Dim writer As IO.BinaryWriter
Dim bufferSize As Integer = 1048576
Dim outByte(bufferSize - 1) As Byte
Dim retval As Long
Dim startIndex As Long = 0
Dim reader As SqlClient.SqlDataReader = command.ExecuteReader()
reader.Read()
stream = New IO.FileStream(DSTFileName, IO.FileMode.OpenOrCreate, IO.FileAccess.Write)
I created a table with column name "description" as varchar(8000). My doubt is if I am not storing 8000 characters in this column, will SQL Server use memory space needed for 8000 characters ? or will it use only the space that needs for my text. ?
I'm in a flat file connection manager editor and I have a flat file where the columns are separated by a space. Does anyone know how to specify a space in the column delimiter option? I've tried {' '} and {s} but these don't work. Not sure what the syntax is for indicating a space. Thanks in advance
Hi, I am declaring a table with 2 column Id, and Name. Id is a integer column but Name is a Varchar. I am declaring Name as VARCHAR(100) you can say that varchar(100) is more than enougf for a name column. But lets suppose ifn futuer user will enter data only till 100 characters but lets suppose i put or declare it as NAME VARCHAR(4000), That means a column requires only 100 memory space but i am giving 4000 to that column So in case of SQL any drwbacks are there for this approach. Meaning my 3900 space will not be used so will it a wrong thing or we can declare it as ,,, it will put no effect??? or what are the drawback??
I have a huge table in it own tablespace (250 GB). I just deleted more thanhalf of the columns in that table, but I didn't see any change in allocatedspace. I did DBCC CLEANTABLE and DBCC SHRINKFILE, but nothing. MS also saysthat CLEANTABLE only reclaims space for columns with text, char datatypes.How can I get space back for integer, decimal, number datatypes? I'm justall out of ideas.
I need a query that checks each row in a column (hardness) to see if the last character is a "space". The column holds numbers (73.45) but is a varchar 50 type. It will always be the 5th character but it would be nice to not make this assumption. Please help!! Thanks.
Hello:I didn't find any documentation that notes save point names are casesensitive, but I guess they are...Stored Proc to reproduce:/* START CODE SNIPPET */If Exists (Select * From sysobjects Where Type = 'P' and Name ='TestSaveTran')Drop Procedure dbo.TestSaveTranGoCreate Procedure dbo.TestSaveTranAsBeginDeclare@tranCount int--Transaction HandlingSelect @tranCount = @@TRANCOUNTIf (@tranCount=0)Begin Tran localtranElseSave Tran localtranBegin Try--Simulate Error While ProcessingRAISERROR('Something bad happened', 16, 1)/*If this proc started transaction then commit it,otherwise return and let caller handle transaction*/IF (@tranCount=0)Commit Tran localtranEnd TryBegin Catch--Rollback to save pointRollback Tran LOCALTRAN --<< NOTE case change--Log Error--Reraise ErrorEnd CatchEndGo--Execute Stored ProcExec dbo.TestSaveTran/*Should receive the following message:Cannot roll back LOCALTRAN. No transaction or savepoint of that namewas found.*//* END CODE SNIPPET */What is really strange, if there is a transaction open, then no erroris thrown. So if you execute as so:/* START CODE SNIPPET */Begin Tran--Execute Stored ProcExec dbo.TestSaveTran/* END CODE SNIPPET */There is no "Cannot roll back LOCALTRAN...." message.Questions:1-)Can someone confirm save point names are case sensitve and this isnot happening because of a server setting?2-)Is this a logic error that I am not seeing in the example codeabove?We have changed our code to store the save point name in a variable,which will hopefully mitigate this "problem".Thx.
I have a table in which there is a column called 'Forecast' of type 'int', that is null for about 25% of the rows in this table. Will null value for this column consume storage bytes?
As I understood, if SPARSE is used on a column, which have many NULL marks, then the storage could be efficently used (we need less spaces to save NULL marks, hence a table which has many NULL marks with SPARSE property needs less storage than the same table, but without SPARSE. I created two table as follow:
/******* Table with Sparse ******/CREATE TABLE Sprstb( unsprsid INT IDENTITY(1,1) NOT NULL, Firstname varchar(20) NOT NULL, Lastname varchar(20) NOT NULL, Tel int NOT NULL, adress nvarchar(60) SPARSE NULL)/***** Table without Sparse*******/CREATE TABLE Unsprstb(unsprsid INT IDENTITY(1,1) NOT NULL,Firstname varchar(20) NOT NULL, Lastname varchar(20) NOT NULL, Tel int NOT NULL, address nvarchar(60) NULL)
I have populated the Sprstb with 5 Milion records. It needs 509,961 MB storage. Then I have copied this table into Unsprstb
SET IDENTITY_INSERT [dbo].[Unsprstb] ON Insert [dbo].[Unsprstb](unsprsid,Firstname,Lastname,Tel,adress) SELECT unsprsid,Firstname,Lastname,Tel, adress FROM [dbo].[Sprstb] SET IDENTITY_INSERT [dbo].[Unsprstb] OFF The Unssprstb need only 466,031MB !
That means the Table with SPARSE column need more storage, Why?
By the way, in table Sprstb column address has 1666198 Null mark (from 5000000)
I've got have a populated table and I want to convert a datetime column so it lists the date only (without the time component) I tried to run this as a script, but returns an error:
update <table> where <column>=convert(datetime,convert(char10),<column>,101))
When I run only this part, it does strip the date of the time component but it becomes a string, and I need this field stored as a datetime field:
I have a column that is frequently updated with an append query. I need to turn all of the incoming positive values into negative values (as well as convert all of the existing). I tried to multiply the column by -1 in the formula option in table design view but couldn't find the proper formula format.
