Available Strategies To Upgrade A Database Schema Without Downtime
Mar 29, 2007
There is a great book on database refactoring that contains a comprehensive set or recipies on how to revise databases that are supposed to be always online and may have various clients that can't be upgraded at the same time. I guess this is a typical case with large databases and I would be surpised if Amazon stops their servers just to move a column from one table to another. The book describes necessary steps for such changes. Basically it's all about creating intermediate database schemas that would be used during transition period.
For example, if we need to move a column from one table to another:
Version 1.
Table A columns: Name, Price
Table B columns: Quantity, Date
Let's say we move Price to table B:
Version 2.
Table A columns: Name
Table B columns: Quantity, Date, Price
The book suggests an intermediate version:
Version 1_2.
Table A columns: Name, Price
Table B columns: Quantity, Date, Price
Additional trigger that will synchronize "Price" columns between A and B.
Version 1_2 can be used by both clients written for version 1 and 2. Software developers don't need to rush their upgrades, transition can last months and include several changes.
This technique requires accuracy in version control management, but looks very good to implement non-interruptible database schema upgrade. I wonder if this is the only option available for data schema upgrade with no downtime. I can't think about anything else - it this how large data warehouses updata their databases?
Hello,I'm upgrading from SQL 7 to SQL 2000 on another box. To minimize thedowntime I would like to1) backup my sql 7 database,2) copy it to the new box with SQL 2000 already installed,3) restore the database on the SQL 2000 box,4) Shutdown my sql 7 database,5) Copy the transaction logs to the SQL 2000 database,6) Restore the transaction logs to the SQl 2000 database,7) Bring up SQL 2000.My only concern with this is restoring the transaction logs that werecreated on SQL 7 to SQL 2000. Do you know if I can do this?Do you see any (other) problem(s) with my plan.Thanks, Scott
Two servers are configured with Windows 2008 / SQL server 2012 utililizing Always-On for HA. We need to upgrade both servers to Windows 2012 / SQL Server 2014 with minimum downtime(Time for Always-On failover). The upgrade to SQL 2014 is straight forward with for minimum downtime.The Windows upgrade(2008 -> 2012) is the problem. From what I have observed and read in blogs.The Windows node to be upgraded must be removed from the Windows cluster before the node can be upgraded to Win 2012.A Win 2008 and Win 2012 node can not reside in the same cluster. If this is true then the only option I can think of is to dump the DB on WIN 2008 server and restore on Win 2012. This is an outage(time it takes to dump and restore).Is there any other method to upgrade these two nodes utilizing Always-On of some other method without downtime?
I have written some software using .NET 2.0, a WinForms app, which uses SQL Server Express as a database. Although I've done my best to set up the database to take into account all of the features, etc., after releasing the first version, I need to make some changes to the database schema, but want to do so without wiping out the entire database for existing users. How do I go about upgrading the database schema for users who are already using the software? I'm thinking I should start by writing a helper app to do so whose sole purpose is to upgrade the schema. The app would be run at install-time, but I don't have any idea how to actually upgrade the schema. I am using C++/CLR .NET, but examples in other .NET languages are more than welcome. Any help is greatly appreciated. Thanks!
We utilise SQL scripts, executed via sqlcmd.exe, to upgrade the schema and common data of our database(s) when we deploy new versions of our software to a production site. At the moment we simply wait until after hours to do the upgrade and ask nicely for all users to not use the system for a while.
Obviously, asking nicely doesn't always work, and there is also the issue of scheduled server tasks and web services / web sites that operate against the database 24/7.
What are our options for putting the entire server (or preferrably just one database) into a semi-offline state so that users and services cannot connect to it while our script connects and performs the upgrade? I imagine there may be several approaches each with their own pros and cons.
If you could point me in the right direction or perhaps mention what strategy has worked for you, it would be greatly appreciated. We perform these upgrades from hundreds of kilometers away via VPNs and Remote Desktop so we can't just unplug the network cable :).
I have a process that restores a production DB, overwriting the existing copy each night. I'd like to keep the solution "up" for as long as possible. And this'll be more important if I want to update it in the day (where there are more queries) too. The nature of queries thrown at the system is that there are about 20 per hour, it's underpinning a reporting system, it's not an OLTP system.
It seems to me I could restore the fresh DB copy into a holding DB, then rename it to the production DB name at the end of the process. The rename process should be pretty much instant.
But I need to think about detecting and waiting for queries to complete on the prod DB, before removing/demoting it (actually, I though to rename it, then reusing it as the next copy to update).
I need a query to find out the server uptime and downtime of the server from MOM database, i don't know in which tables MOM actually stores this infomation.
I need this very urgently.
Thanks in advance
You can use this code to find out the information stored in the MOM tables:- ############################################################################ create PROC [dbo].[SearchMyTables]
Hi, trying to affect schema changes on an SQL DB using however Im encountering a number of errors when trying to do this. Changes are are affected by running a consecutive number of scripts against the DB, see example of script below:
-- PREVIOUSDBVERSION=2.8.2.5 -- The above line must be present in all Upgrade SQL files to catch any gaps in the upgrade chain. -- --ALL -- SET NUMERIC_ROUNDABORT OFF GO SET ANSI_PADDING, ANSI_WARNINGS, CONCAT_NULL_YIELDS_NULL, ARITHABORT, QUOTED_IDENTIFIER, ANSI_NULLS ON GO IF EXISTS (SELECT * FROM tempdb..sysobjects WHERE id=OBJECT_ID('tempdb..#tmpErrors')) DROP TABLE #tmpErrors GO CREATE TABLE #tmpErrors (Error int) GO SET XACT_ABORT ON GO SET TRANSACTION ISOLATION LEVEL SERIALIZABLE GO BEGIN TRANSACTION GO
PRINT N'Dropping Stats from A DB' GO DECLARE @tblname sysname, @statname sysname, @sql nvarchar(2000)
DECLARE c CURSOR FOR SELECT object_name(id), name FROM sysindexes WHERE INDEXPROPERTY(id, name, 'IsStatistics') = 1
OPEN c FETCH NEXT FROM c INTO @tblname, @statname
WHILE @@FETCH_STATUS = 0 BEGIN SET @sql = 'DROP STATISTICS [' + @tblname + '].[' + @statname + ']' PRINT @sql EXEC (@sql) FETCH NEXT FROM c INTO @tblname, @statname END
CLOSE c DEALLOCATE c GO IF @@ERROR<>0 AND @@TRANCOUNT>0 ROLLBACK TRANSACTION GO IF @@TRANCOUNT=0 BEGIN INSERT INTO #tmpErrors (Error) SELECT 1 BEGIN TRANSACTION END GO
The database I am upgrading has a growth limit of 2GB, when running the scripts against the DB two things are happening.
1) DB size begins to increase rapidly and eventually breaches 2GB mark 2) LDF file swells to upto 3GB
My first thought is that the schema changes are gathering in the log file and not being commited to the DB, or that there is some kind of open transaction that the script is constantly trying to execute over and over.
Would like to point out im a novice SQL user, if the information above is sketchy please dont hesitate to say and I will endeavour to provide you with more accurate information.
Locally I develop in SQL server 2005 enterprise. Recently I recreated my db on the server of my hosting company (in sql server 2005 express).I basically recreated the tables and copied the data in it.I now receive the following error when I hit the DB:The 'System.Web.Security.SqlMembershipProvider' requires a database schema compatible with schema version '1'. However, the current database schema is not compatible with this version. You may need to either install a compatible schema with aspnet_regsql.exe (available in the framework installation directory), or upgrade the provider to a newer version.I heard something about running aspnet_regsql.exe, but I dont have that access to the DB. Also I dont know if this command does anything more than creating the membership tables and filling it with some default data...Any other solutions/thought on what this can be?Thanks!
Hello everybody!I'm using ASP.NET 3.5, MSSQL 2005I bought virtual web hosting .On new user registrations i have an error =(The 'System.Web.Security.SqlMembershipProvider' requires a database schema compatible with schema version '1'. However, the current database schema is not compatible with this version. You may need to either install a compatible schema with aspnet_regsql.exe (available in the framework installation directory), or upgrade the provider to a newer version. On my virtual machine it work fine but on web hosting i have an error =(What can you propose to me?
Dear all,I'd like to explore more about indexing strategies and would appreciatea reference to some good resources this regard including book names.Best regards
hello, what are the strategies when designing tables that needspaging?in the past i used to useselect top 200 * from tablewhere id not in (select top 100 id from table)with SQL 2005, would u guys recommend using CTE and/or ROW_NUMBER?or any other advice?thanks
I have been somewhat dismayed by the lack of information available relative to testing SSIS solutions. At a time when both managed code and db code are getting tooling (through VSTS) to help drive an automated testing mentality into developers everywhere - there seems to be little out there for those using SSIS. I have seen people talk about having SSIS packages that test the actual packages - there is always the manual way to write the tests in something like C# and then programmatically invoke the package and manually validate the results. Anyone out there have approaches that are working for them?
Im about to start a project that will be hosted by a third party web host. What is a common way to backup your database and have the backup saved ? The data may end up being several 100 MB of user settings, text etc (blog type stuff). If the DB gets to be several 100MB, then does making a backup and ftping it offsite sound reasonable ? Does ftp bandwidth usually count against your overall bandwidth usage ?
I'm trying to flesh out a good queue table design with our dev team.So here is a general overview of the scenario. First an application will hit a WebAPI and grab any updates to Content and store those ID's in SQL (queue table). Next is the fun part, different multi threaded apps will process ID's from the queue. One app will make updates to the data in a different SQL DB while the other will update an index (likely Elastic).
Obviously, we don't want multiple threads working on the same items. One strategy could be to use UPDLOCK & READPAST query hints. However, I'm not sure about the reliability or performance of this solution. I just started looking into setting up a service broker but that would be completely unfamiliar territory for me. Also I can see how a broker might work well within the instance but how would that work with the application making updates to Elastic?
I am looking for some published paper regarding database performancetunning performance strategies. This is for academic purpose so itneeds not to be any commerical database specific. It will be evenbetter if the paper has some kind of methods to quantify/measureperformance. Has anyone come across with any interesting paper aboutthis?Thanks,ewong
My SQL Server 2005 Database was down last night.From logs I can find out below details only.
Code Block Date 11/30/2007 1:01:34 AM Log SQL Server (Archive #1 - 11/30/2007 1:01:00 AM) Source spid4s Message SQL Server is terminating in response to a 'stop' request from Service Control Manager. This is an informational message only. No user action is required.
Now I want to find out root cause for this.Who has stopped this service.And If services was stopped automatically then which program/service is resposible?
So Please suggest me how to do root cause analysis for this downtime.
I recently added a new user to my database. Now I want to delete that user, but I keep getting the error above. What do I need to do to delete my recently added user?
I am trying to create an alert on my new SQL 2005 box to email alert/notify when/if the service ever stops, etc. Is there an existing alert I can use or do I have to script one, and if I do - how would I do it?
I am trying to create an alert on my new SQL 2005 box to email alert/notify when/if the service ever stops, etc. Is there an existing alert I can use or do I have to script one, and if I do - how would I do it?
An error has occurred during report processing. (rsProcessingAborted) Query execution failed for data set 'TestID'. (rsErrorExecutingCommand) For more information about this error navigate to the report server on the local server machine, or enable remote errors
The log file reads:
---> Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Query execution failed for data set 'TestID'. ---> System.Data.SqlClient.SqlException: SELECT permission denied on object 'TableID', database 'Database', schema 'dbo'.
***Background***
General Users got an error message when trying to access any reports we have created. All admin have no problems with the reports. Users (Domain Users) are given rights (Browser) to the reports and the Data Sources (Browser) and yet cannot view the reports.
An error has occurred during report processing. (rsProcessingAborted) Cannot create a connection to data source 'DS2'. (rsErrorOpeningConnection) For more information about this error navigate to the report server on the local server machine, or enable remote errors
I'll add this from the report logs...
w3wp!processing!1!3/20/2007-11:43:25:: e ERROR: Data source €˜DS2€™: An error has occurred. Details: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source €˜DS2€™. ---> System.Data.SqlClient.SqlException: Cannot open database €œDatabase€? requested by the login. The login failed. Login failed for user €˜DOMAINUsername€™.
The user has rights via a local group to the report and data source (Browser rights) and the local group has been added as a SQL login.
I gave rights to the databases themselves instead of just to SQL and the error changed (Ah-ha...progress, but why!?!?)
I'm preparing a checklist for myself before getting ready to migrate from 2005 to 2012. Our largest database is a nice one at over 250GB. I'm thinking my best bet to minimize any downtime would be to Restore the DB (NORECOVERY) on the new server and keep rolling it forward with the transactional logs. Eventually I'll need to bring the old DB offline and do one last backup and apply that one to the new server but that should be a small time frame given the whole process could take several hours.
1. I am looking data base solution for highly critical application. what type of HA/DR solution will suite for this.(always on, transactional , log shipping or mirroring). make sure all secondary nodes should be identical to production .
2. for any need to upgrade /patching /maintenance work, Â production should not be able to down. No downtime 100%.
I need to export a database, x, of a server, X, to another database, y, of a server, Y and I need export the database schema only, not include the data. Does anyone know how to do that? Many thanks for replying.
Trying to get my hands around all the new security features of SQL Server 2005. In Management Studio did something I don't know how to undo. I added a database role ReadOnlyRole and clicked the box next to db_datareader in the owned schemas box. Then I tried to remove the ReadOnlyRole and could not. How do I undo what I did? Is it possible?
The below is the TSQL that generates the my issue.
Use [master] go create database [test] go
USE [test] GO
CREATE ROLE [ReadOnlyRole] GO
USE [test] GO
ALTER AUTHORIZATION ON SCHEMA::[db_datareader] TO [ReadOnlyRole] GO
I'm trying to run a query to check the downtime in production lines, but if a line has assigned more than one cause for the downtime it repeat the info for each cause.
This is the code.
SELECT D.Line AS Line, D.ProductionLine AS ProductionLine, D.Shift AS Shift, D.DownTime, CONVERT(VARCHAR(10), D.DatePacked,101) AS DatePacked, AssignedDowntime, (D.DownTime - AssignedDowntime) AS NOASSIGNED, R.Enviromental,R.Equipment, R.IT_Systems, R.Material_External,R.Quality,R.Material_Internal, R.Method,R.PreProduction,R.People FROM ( SELECT Line, Shift, DatePacked, SUM(Cast(Downtime AS INT)) AS AssignedDowntime,
[Code] ....
I'm expecting that if is more than one "Down Reason "it will include in the same line. At this moment if i have more than one reason it create a line for each one for example:
If i have a total Downtime of 50 minutes and they are assigned 10 for itequipment, 30 by testequipment and 10 assigned to quality issues i will have and output like this:
I have a table (named table1) with 20million rows. It takes around 11 minutes to apply the primary key to this table. There are some tables with over 100 million rows so based on the previous time if my calculations are correct it will take close to an hour apply this primary key for tables with around 100 million rows.
My current solution is to create another table (named table2) with no indexs or primary keys. Pump over only like 5 days worth of data, then apply the primary key. Then have a script that will eventually populate table2 with the rest of the data gradually. When I say gradually I mean like insert like every 100k per hour or something. Keep in mind this table2 is heavily updated with new records.
here is my problem. I have to rebuild database after crash. there is no backup. So I did a bcp to get data from. But I do not have the original database so my question is how to get the full schema off the data base, tables,colomns,stored procedures etc...