Why Does A Timeout Make Sense? (Reading From DTS Buffer Timed Out)
Aug 28, 2007
I've looked at the other threads on this topic, and don't see an answer to the following question:
Why should an error destination time out waiting for error rows?
I'm using SQL Server Destinations both for my staging tables and for my "Error Staging" tables. Yet it seems that these are timing out if the package runs a long time without any error rows. This leads to two questiosn:
Why should an error destination time out waiting for error rows?
I can solve this by setting the timeouts to some very large number. But, is there a better way to do this?
Right now, if the package takes five minutes, I need to set the timeouts to longer than five minutes. That does not sound like a good idea.
I have inherited a SQL 2005 server with a few small databases on it. There's a maintenance plan here that doesn't seem to make a lot of sense to me. Can anyone comment:
Every Sunday at 4:00 AM
1. Reorganize index on All user database Tables and Views - compact large objects. 2. Rebuild index on local server connection, All user databases, Tables and view, Original amount of free space. 3. Shrink database. All user databases. Limit 100MB.
I'm confused a little about item 3. Won't a shrink be kind of useless after all of the work that goes on in steps 1 and 2. When I ran this manually, the transaction logs jumped significantly.
l've written a cursor to koop through a table and then insert the last 100 records into a table.Reason why l want the last 100 records is to monitor and log the last 100 trans avery hr or so.
-- Declare the variables to store the values returned by FETCH. SET ROWCOUNT 100 DECLARE @customer_No char(15), @loan_No char(12), @date_Issued datetime , @maturity_Date datetime , @status int
DECLARE loan_cursor CURSOR FOR SELECT customer_No, loan_No, date_Issued, maturity_Date, status
FROM loan
OPEN loan_cursor
-- Perform the first fetch and store the values in variables.
FETCH NEXT FROM loan_cursor INTO @customer_No, @loan_No, @date_Issued , @maturity_Date, @status,
-- Check @@FETCH_STATUS to see if there are any more rows to fetch.
WHILE @@FETCH_STATUS = 0 BEGIN
-- This is executed as long as the previous fetch succeeds. FETCH NEXT FROM loan_cursor INTO @customer_No , @loan_No , @date_Issued , @maturity_Date , @status END
CLOSE loan_cursor
DEALLOCATE loan_cursor;
insert into Loan1 (customer_No, loan_No, date_Issued , maturity_Date , status )
select @customer_No, @loan_No, @date_Issued, @maturity_Date, @status FROM loan ORDER BY date_Issued desc;
I need any one's advice/imput on this...PLEASE!My computer will now begin the process of taking all the MS Access (NativeJet Engines - x30 total departmetns) and put the tbles/BE on SQL Server 2005and the Ms Access FE on MS Sharepoint.This is the kicker, say 20 out of the 30 (ball park) was created by oneperson and that is their whole job function was to create/maintain a QAtracking system and more.The person who created the 20 out 30 only knows intermediate ms access andsome vba, but NOT sql or net conversions (visual studio - all the differentlanguages), so the IT director asks me (I develop in MS Access andintermediate in VBA and can create web sites using publisher, front page andHTML) he asks me and this other person if we want to take on the challenge ofhelping him and the other IT guy in the conversion process of all of thesedb's.What does this do the developers who developed and still maintain thesecurrent 30 ms access db's, well you guessed it, it now takes all that hardwork that those developers did and still do (they still add more forms,updates) and it NOW takes the databases owners away from them and grant itnow to the person (s) who will maintain SQL Server 2005 ( I hope will be aDBA)???Is this true, once all the databases are converted, the owners will no longerbe able to go behind the scenes in tables, queries,etc.... It will now be inthe hands of a DBA?You know the funny thing is the IT Director wasn't even sure if he was goingto hire a DBA, who in the heck will maintain all of those db's on the server?There is only one other guy and he certainly does not have the training orskills or TIME.MY POINT QUESTION IS:when these conversion take place like this at a company, most of the time thems access dbs that have now be put into sql will now take the ownership awayfrom the owner (they cannot develop no more, unless they are sql friendly/dba)and put all of that into one persons hand (DBA) to maintain and development??????--Message posted via SQLMonster.comhttp://www.sqlmonster.com/Uwe/Forum...eneral/200606/1
Thanks to several guys here, I now understand how SQL Server configuration option works... Pretty nifty stuff.
Now, I'm trying to see if I can configure the Server property of the Connection Manager that holds the information for where my configuration table is. I thought about this and tried it, but it doesn't work. Then it occurred to me, this may not make sense to try to do because it is like the question, "what came first? the egg or the chicken?"
I am new to SSIS. i am trying to port database from SQL SERVER 2000 to 2005. i am using "Transfer SQL Server Objects" for this. i am just trying to move one object for testing wether it works or not. and it is not working. i am getting this error.
[Transfer SQL Server Objects Task] Error: Execution failed with the following error: "ERROR : errorCode=-1071636471 description=An OLE DB error has occurred. Error code: 0x80040E37. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E37 Description: "Invalid object name 'dbo.consta_AE'.". helpFile=dtsmsg.rll helpContext=0 idofInterfaceWithError={8BDFE893-E9D8-4D23-9739-DA807BCDC2AC}".
Both DBs are on seprate machines. of you need any more info please let me know. that would be great help.
We started to develop a datawarehouse solution for client back in December 2008 based on SQL 2008. We are convinced that we can use some of the new feature included in the new version which is the reason we we chossed to go down this path.
Due to the delay of the next version we have some question that I would like to hear you opinion on.
The estimatet "go live" date is during spring (march/april)
Is it wrong to contiue the development on the 2008 version?
We would need to run Release candidate at the customer site until the product is released. Is there any major changes coming up that are already known?
So far in the development we have had no big problems with releasecandidate.
We are importing a very small subset of a big desktop database into a CE database on a mobile device for an occasionally connected application. The idea is that the mobile device can use this CE database as a fall back database in case we are not connected.
The database is a very simple list of barcodes.
Basically a single field as primary key
EAN13 bigint
When we import 200K rows (yes we have quite a lot of them). The database is 7MB!!!! A bit big I would say since 8 bytes times 200.000 is only 1.5 MB. Where does the extra space come from?
I'm troubleshooting a stored procedure that suddenly decided to stop working. I narrowed down the problem to the last part of the stored procedure where it selects data from a temp table and inserts it into a physical table in the SQL2000 database.
I keep receiving the following error:
Server: Msg 8115, Level 16, State 8, Line 140 Arithmetic overflow error converting numeric to data type numeric.
The data values all appear to be correct with none of them seeming to be out of precision, but I keep getting the error. I've tried casting all the values and it didn't work. It executes w/o error when I comment out that particular insert. I just don't get it.
-- Input: @SPVId - SPV we are running process for -- @Yes - value of enum CCPEnum::eYesNoYes (get by lookup).
-- Output: Recordset (temp table) of Collaterals that are eligible for MV Test (#MVTriggerInvestments).
DECLARE @Yes INTEGER EXEC @RC = [dbo].CPLookupVal 'YesNo', 'Yes', @Yes OUTPUT IF (@RC<>0)BEGIN RAISERROR ('SP_OCCalculationMVTriggerTest: Failed to find Yes enum', 16, 1) WITH SETERROR END drop table #MVTriggerInvestments BEGIN
SELECT dbal.SPVId, dbal.CusipId, dbal.GroupId, @dtAsOfDate AS AsOfDate, dbal.NormalOCRate, dbal.SteppedUpOCRate, dbal.AllocMarketValue AS MarketValue, dbal.NbrDays, dbal.PriceChangeRatio
INTO #MVTriggerInvestments
FROM DailyCollateralBalance dbal
JOIN CollateralGroupIncludeInOC gin ON dbal.SPVId = 2 AND gin.SPVId = 2 AND dbal.AsOfDate = '2006-04-16' AND @dtAsOfDate BETWEEN gin.EffectiveFrom AND gin.EffectiveTo AND dbal.GroupId = gin.GroupId AND gin.IncludeInOC = @Yes
END select * from #MVTriggerInvestments print 'end #1' --select * from #MVTriggerInvestments --looks ok
-------------------------------------------------------------- -- 2) Calculate Weighted Average Price change ratio Market Value (by Group): -- PCRMV - Price Change Ratio Market Value --------------------------------------------------------------
-- Input : Recordset of collaterals (having New/Old prices, MarketValue defined) -- Output: Recordset Aggregated by Group (#GroupOCRate) drop table #MVTriggerGroup BEGIN
cast([dbo].fn_divide_or_number (B.PriceChangeRatioMarketValue, B.MarketValueForPeriod, 0.00) as numeric(12,9)) as PriceChangeRatio,
CAST (0 AS NUMERIC(12,9)) AS OCRate, CAST ('' AS VARCHAR(6)) AS OCRateType, CAST (0 AS NUMERIC(18,2)) AS DiscMarketValue, CAST (0 AS NUMERIC(18,2)) AS InterestAccrued
INTO #MVTriggerGroup
FROM ( SELECT SPVId, AsOfDate, GroupId, NormalOCRate, SteppedUpOCRate, cast(SUM(MarketValue) as numeric(18,2)) AS MarketValue
FROM #MVTriggerInvestments GROUP BY SPVId, AsOfDate, GroupId, NormalOCRate, SteppedUpOCRate ) A --works up to here
JOIN (SELECT SPVId, cast(SUM(AllocMarketValue) as numeric(18,2)) AS MarketValueForPeriod , cast(SUM(AllocMarketValue * PriceChangeRatio) as numeric(18,2)) as PriceChangeRatioMarketValue, GroupId
FROM T_DailyCollateralBalance WHERE SPVId = 2 AND AsOfDate between '2006-03-17' and '2006-04-15' AND IsBusinessDay = 1 GROUP BY SPVId, GroupId ) B
ON A.SPVId = B.SPVId AND A.GroupId = B.GroupId
END print 'end #2' --------------------------------------------- -- Calculate OCRate to apply for each group. --------------------------------------------- BEGIN UPDATE #MVTriggerGroup SET OCRate = (CASE WHEN ((PriceChangeRatio < 0) AND ABS(PriceChangeRatio) > (0.55 * NormalOCRate)) THEN SteppedUpOCRate ELSE NormalOCRate END), OCRateType = (CASE WHEN ((PriceChangeRatio < 0) AND ABS(PriceChangeRatio) > (0.55 * NormalOCRate)) THEN 'stepup' ELSE 'normal' END) END print 'end #3' ------------------------------------- -- Calculate discounted Market Value ------------------------------------- UPDATE #MVTriggerGroup SET DiscMarketValue = MarketValue / (1.0 + OCRate * 0.01) print 'end #4' --------------------------------- -- Insert data from temp tables --------------------------------- -- 1) select * from #MVTriggerInvestments
I am wondering if there is any sense to create indexed views on single table? I simple want to improve the report query performance as most of the reports data are from a single table. As views most of the time are created as for joined across tables.
Thank you very much for your advices and I am looking forward to hearing from you shortly.
In short, we have started deploying Office 2007 to our users and Excel is currently the only client we use to interact with our AS2005 cubes.
A few users have reported issues (which I've verified), but the explanation in the KB article doesn't make any sense to me. These reports were originally developed in Excel 2003 and when opening them up in Excel 2007, we'll see a message saying that Excel found unreadable content in the .xls file and after clicking 'Yes' to recover contents of the workbook, we then receive a message that a PivotTable report was discarded due to integrity problems. If I opened up this report in Excel 2003, I don't receive these errors or messages.
Per the KB's explanation (http://support.microsoft.com/default.aspx/kb/929766):
This issue occurs if the following conditions are true: €¢The workbook contains a PivotTable that uses key performance indicators (KPIs).€¢The KPIs are created in the Analysis Services Business Intelligence Development Studio.€¢One or more of the KPIs have an expression in the Current Time Member property. Now, we are running 2005 Standard Edition with no SP, but will be deploying SP2 in a few days. Our cubes do not have any KPIs defined. Can I even define KPIs if we are only running Standard Edition?
Time out occurred while waiting for buffer latch type 2,bp 0x18b7d40, page 1:11558916), stat 0xb, object ID 9:1842105603:2, EC 0x5862D9C8 : 0, waittime 300. Not continuing to wait.
Hi all I've a page which shows data (25-30 entries) that are retrieved by an SqlDataSource. The query is very simple since i don't have to join any table.unfortunately sometimes I get the following error message. It's strange, since the page often works fine but the error occurs a couple of times per day.Who has an idea of how I could solve this problem?
I actually work in an organisation and we have to find a solution about the data consistancy in the database. our partners use to send details to the organisation and inserted directly in the database, so we want to create a new database as a buffer database to insert informations from the partners then make an update to the main database. is there a better solution instead of that?
I encountered the following error while attempting to preview an RDL report I was developing in VS2010 using SSDT:"The size necessary to buffer the XML content exceeded the buffer quota"
We have a set of reports with same header section in all the reports. So while developing a new report i used to copy that header section to the new report with same dataset names (without any change) , but while rendering the report it is throwing error " The size necessary to buffer the XML content exceeded the buffer quota".
I have a master package that executes a series of sub packages run from a SQL Agent job. One of those sub packages has been stable for a week, running at least once per day, but it just failed despite having been run once already today with the same set of input data.
There were a series of errors showing in the event log for the Execute Package Task starting with "Buffer Type 15 had a size of 0 bytes.", then "The buffer manager failed to create a new buffer type.", then "The Data Flow task cannot register a buffer type. The type had 32 columns and was for execution tree 3.", then "The layout failed validation." and finally "Error 0xC0012050 while loading package file "C:[Package].dtsx". Package failed validation from the ExecutePackage task. The package cannot run.".
SQLIS.com reports the constant for the error code as DTS_E_REMOTEPACKAGEVALIDATION ( http://wiki.sqlis.com/default.aspx/SQLISWiki/0xC0012050.html ).
I then ran the package on my dev machine in BIDS and it worked fine, so I re-ran the job on the server and this time that package executed ok, but another one fell over but did not put anything in the event log.
I'm experiencing a completely random warning from any given row count component within any given data flow task. It occurs sporadically. Whilst distracting, I don't see any adverse effects to the data after the packages complete. Can someone weigh in on this warning and let me know if it is indeed benign or what I maybe able to do to fix it?
Here's the warning:
"A call to the ProcessInput method for input 75997 on component "CNT Rows sent for STG table" (75995) unexpectedly kept a reference to the buffer it was passed. The refcount on that buffer was 4 before the call, and 5 after the call returned."
I have a few questions related to using CONTAINSTABLE in a query that I hope someone can help with.
I am working on a project to add document search capabilities to my companies product using fulltext indexing. Part of this requirement is an ability to breakdown the component parts of of the search query and provide information on *why* documentX ranked higher than documentY. This is a bit convoluted, but taking this (very simple) example - the user wishes to search for 2 skills - "HTML" and/or "XML". The generated query looks a little like :-
Select DOC.DOC_ID, RANK1.RANK, RANK2.RANK, RANK3.RANK from DOCS DOC inner join CONTAINSTABLE(docs, doc, 'HTML AND XML') as RANK1 on RANK1.DOC_ID=DOC.DOC_ID inner join CONTAINSTABLE(docs, doc, 'HTML') as RANK2 on RANK2.DOC_ID=DOC.DOC_ID inner join CONTAINSTABLE(docs, doc, 'XML') as RANK3 on RANK3.DOC_ID=DOC.DOC_ID
This returns the "overall" rank, and a rank for the 2 component parts, so I can say this doc ranked XXX overall because it scored "rank1" for HTML and scored "rank2" for XML etc....
My question on this part is about the values for the "overall rank". If the query contained an OR it always seems to return the highest of the "rankX" values, and if it doesnt, it returns the lowest. e.g. for the example for java and word and excel and access - the overall ranking is 2 , java=36, word=2, excel=16 and access=36 for java and word or excel and access - the overall ranking is 16 , java=36, word=2, excel=16 and access=36 for (java and word) or (excel and access) - the overall ranking is 16 , java=36, word=2, excel=16 and access=36
So in the first example, regardless of what the other values are, the rank returned is always 2 (the score for "word"). My resultset has 100ish rows, all with a rank of < 5 for word, but all with ranks of 18-100 for the other 3 values - yet the "overall" rank always matched the "word" rank.....?? This doesnt feel right to me somehow, I would expect a different value as if the document ranked really highly for one value but low for the other, it doesnt feel right the value is clamped to the lowest? Or am I just understanding it wrong? If I use "freetexttable" the overall rank is a little more meaningful - but unfortunately I also need to use weighting, which brings me to my next question . . .
This question is about rankings returned from the ISABOUT function. In the following example, select * from documents as DOC inner join containstable(docs,doc,'project') as doc0 on DOC.DOC_ID=doc0."key" inner join containstable(docs,doc,'ISABOUT (project weight (1.0))') as doc1 on DOC.DOC_ID=doc1."key" inner join containstable(docs,doc,'ISABOUT (project weight (0.5))') as doc2 on DOC.DOC_ID=doc2."key" inner join containstable(docs,doc,'ISABOUT (project weight (0.1))') as doc3 on DOC.DOC_ID=doc3."key" inner join containstable(docs,doc,'ISABOUT (project weight (0.0))') as doc4 on DOC.DOC_ID=doc4."key" order by doc0.rank desc
The values I get from the doc1/2/3/4.RANK columns dont seem right. In this example,
doc0.rank = 133 doc1.rank = 150 doc2.rank = 330
doc3.rank = 924
doc4.rank = 0
These values dont make any sense to me, as the rank seems to go UP when the documentation on ISABOUT says it goes down (I think it says somewhere the calculated rank is multiplied by the weight?). Once again, is there something I missed or am I understanding it wrong?
Thanks in advance for any help into understanding the whys of this...
How can I know my sql server 2005 express service pack? The same goes for SSMSE. I can't find these information in the about box.
What about intelli sense for queries in SSMSE? Is it there yet or should I wait for sql server 2008 for dev. (http://channel9.msdn.com/Showpost.aspx?postid=387069)? I haven't seen the video but does anyone know any details? When it will be available, is it part of the sql server 2008 CTP, will it be available in the Express edition?
The scenario is as follows: I have a source with many rows. Each row has a column called max_qty_value. I need to perform a calculation using another column called qty. This calculation is something similar to dividing qty/(ceiling) max_qty_value. Once I have that number I need to write an additional duplicate row for each value from the prior calculation performed. For example, 15/4 = 4. I need to write 4 rows to the same target table as in line information for a purchase order.
The multicast transform appears to only support fixed and/or predetermined outputs. How do I design this logic in SSIS to write out dynamic number of rows to a target table.
We have an SSIS package that was created to migrate data in from a few production databases. The steps for the package are as follows...
backup databases on server 1 (prod database server) restore database to SSIS server (server 2) . truncate worker tables in SSIS server's (server 2) Main DB database. copy data from restored db tables to working db tables ( database to database) Start Multiple threads (15 ) and run steps from here in parrallel Combination of Data flow tasks and SQL scripts and Stored procedures used to flatten data out and combine data for reporting purposes.
The average run time is 8 hours.
the issue we are seeing is this, the package will fluctuate in run times from 4 hours to over 11 with no change in the data or the underlying SSIS package. We have looked for any changes or things that would effect this but have not found anything that changed...
Also, certain steps are running shorter while others double in time. there doesnt seem to be any rhyme or reason to this behaviour. The server is x64 12GB of RAM 2 dual core 3.2Ghz.
Please let me know if you need any more information or specifics...
the only thing I have seen so far that looks out of place is Tempdb has one of its files that is 20+GB.
I've been programming with SQL 7 for about a year and my company has finally decided to go SQL 2k5.
I've come accross a really irritating error when writing to the DB via ADO in ASP pages. I have a column in a table that is auto-incremental.
In SQL server 7 you just make an ADODB.Command object and enter the SQL query 'insert into table (columns) values ('val...') now for SQL 7 I can completely leave out the auto-incremental column (called 'ErrorNo') and simply specify the other columns and values in my insert query. e.g. where my table is called master_error:
ErrorNo int identity (1, 1) not null ,ReportedBy char(10) ,ExpectedFixDate datetime
With ErrorNo being an auto-incremental identity, my query would be
INSERT INTO master_error (ReportedBy, ExpectedFixDate) VALUES ('Ben','01 Sep 2007')
this works perfectly with ADODB.Command when writing to SQL Server 7 from IIS 5.0
however when I execute the exact same command on the exact same table using ADODB.Command writing to SQL Server 2005 from IIS 6.0 I get an 'error 500 internal server error'
I thought perhaps SQL 2005 might have different syntax so I typed the query directly into SQL Server 2005's version of query analyser and guess what... it worked fine.
I can't tell where the error lies. I find it hard to beleive that the error is in the code of my ASP page as it works perfectly against a sql 7 db.
I am getting an error message when i am trying to open my application which is designed in VB and is connected to a remote "SQLSERVER".
Error in Mian Description:[Microsoft][SQL Native Client]Query timeout expired Number: 2147217871
How to solve the issue, I am frequently getting this error message. Is the issue related to a Network problem or problem with the Execution of the query by "SQLSERVER" or is it a combination of both.
It would be a great help if some one can help me out in solving this isssue.
Hi, i have a counter on my page that gets the number of users online, i've made it possible by adding a column that gets value 1 when user login and value 0 when logout. Also the page have a frame that refreshs every 4 minutes and updates the last activity column on my users table.The problem is when session timeouts due to closing the browser of connection down, etc, i mean, when people don't logout with the logout button.I've tried to put code to that in global.asx in session_end but seems not to be working. I've tought that if i can have a page running a stored sql procedure that gets users with value 1 and compares his last activity date with current date and put it to 0 in case of inactivity it can be the solution, the problem is that i don't know how to do it. How can i have a sql procedure running every 5 minutes on server side on a remote host? I think that this is my question. If you have another solution to my problem i'll be glad to hear it, if you do not understand i can explain better.Thanks in advance.
I'm using some db replication tool written in JAVA. Where in the config file I have entered db driver, db url, sa and password. This tool lies in ubuntu. I couldn't able to connect to MS SQL server.
im getting this as a part of the connection timed out message, while connecting from my node to sql server 2000 instance running on windows 2000 server.
today i get one issue from production. Here is the description
1) when one user try to load the large volume of report (SSRS report with 2.0 million records), the other user get the timed out issue?
My SSRS report populate data from the same table and other connected user trying to insert/update the record into same table and they get timed-out error?
I am not sure but SSRS reading data only read-only & forward only mode. right? Why the other users get the timed-out issue?
I am having a problem with an application that does not kill timed out connections. This is normally not an issue, but when something causes the timed out connections to build up, it stops the frontend from working correctly. The frontend developers are trying to figure out how to change their code to check for and drop timed out connections at the application. Until then, I need a way to check for timed out connections at the database and drop them there via a job that will run every 10 minutes or so. I have to make sure that only timed out connections are dropped and not active ones. Any suggestions?
I have an SSIS package that looks for the existence of a file (using Konesans file watcher) on the network to begin processing. I am trying to find a way to terminate/shutdown the entire process if that file is not out there by say 7:30 AM. I think I could set a timeout on the filewatcher task, but was hoping there was another tool I could use. (If the process had to be restarted late a timeout would still countdown the same period wouldn't it? We'd still need the job to stop by 7:30.)
More background. The package, once deployed, will be called from a command line script that is started by a scheduled task normally, or manually in the event of late availability.
