Better Key, More Efficient Indexing
I have a complex query that uses UNION and I want to make this query use indexes for faster execution. Code:
View Complete Forum Thread with Replies
Related Forum Messages:
Efficient Way Of Mass Indexing?
I have been working on a program that will populate and index a database. The populating doesn't take too long, but the indexing does. My question is: Is there a better approach to indexing this table than the one I'm using right now? I'm doing this through QtSql. headers is a QString array of 48 column names I want to index. for (int i = 0; i < headers.size(); ++i) { q.exec("ALTER TABLE mysqltable ADD INDEX(" + headers[i] + ");"); } example run for 20000 rows: viper,david $ time ./main -r 20000 Database Connection Established 0.520u 0.060s 0:21.27 2.7% 0+0k 0+0io 0pf+0w
View Replies !
What's The Most Efficient Way?
I've done a database scheme and wonder what the most efficient way of storing certain entries would be. The site will have news, reviews and tutorials and these will be under the same categories. So I'm wondering what the best way to do design the database scheme would be. Have one table like I've done now in the "content" table with the posibilty to differentiate the entries with the "post_type" field or have three different tables? Below is the table "content". Field Type Null Default post_id int(10) No post_date datetime No 0000-00-00 00:00:00 post_text text No post_title text No post_cat_id int(4) No 0 post_description text No post_name varchar(200) No post_type int(4) No 0
View Replies !
How Efficient Is In() , And How Much Is The Most That You Should Put In An In()?
I am just wondering as to the efficiency of the in() function in MySQL. select field1,field2,...,fieldn from table where id in(1,2,3,4,5,6, ... , n); assuming: id is an indexed field. table is a VERY big table (100 000+ - 1 million+ records) what do you think is the largest number of values you could pass to the in() function without completly flattening your server?
View Replies !
Most Efficient YES/NO...
I) Table_A has fields Data_1, Data_2, Data_3, and Data_4; I need to determine whether value 'X' is present (at least once) in any of the Data_N fields. All I need is a YES/NO answer: YES, 'X' is present (at least once) in one of the Data_N fields of Table_A [Record number(s) not required!], or NO, 'X' is not present in any of the Data_N fields in any of the records in Table_A. II) Similar scenario, but this time Table_A has fields RecID, Data_1, Data_2, Data_3, and Data_4. I need to determine the RecID (none, one, or more) of every record that has value 'X' in any of the Data_N fields. I'm looking for the query that will be fastest and most elegant to implement.
View Replies !
Quickest And Most Efficient
i run an online game. I want to give each user a ranking, based on how high thier score is.If for example, i had 20,000 players, i dont want to have to update them all one by one, it may strain the server and take a long time. Is there another way i can assign a rank number (rank 1 has the top score and so on)
View Replies !
Efficient UPDATE
I have a table with the following structure; CREATE TABLE my_table ( id_1 int(11) NOT NULL , id_2 int(10) NOT NULL , stauts tinyint(1) NOT NULL DEFAULT 0 , PRIMARY KEY (id_1) ) Engine =InnoDB'; The table currently has arround 100,000 entries. When I try to run variations of the following statement it is taking around 4 seconds per query; UPDATE IGNORE my_table SET id_1 = 74240, id_2 = 5 I need it to be running a lot faster than 4 seconds per query as I need to update upwards of 100,000 records a day! My server is fairly beefy, a 3 gig dual core opeteron and is generaly running below 1.0 load.
View Replies !
Efficient Tables.
I am starting out on a project where I need to store GPS information. The data consists largely of a series of "Points" each consisting of a longitude,latitude and elevation. On a typical "route" there could be hundreds of points. My question is how can I efficiently store this information. It does not sound sensible to normalise this and add hundreds of rows to a table for each "route". Sample data is along the lines of : - <trkpt lat="54.016942977905273" lon="-1.4903640747070313"> <ele>82.330078125</ele> <time>2006-09-03T07:35:41Z</time> </trkpt> - <trkpt lat="54.016938870772719" lon="-1.490332055836916"> <ele>0</ele> <time>2006-09-03T07:35:42Z</time> </trkpt>
View Replies !
Efficient Storage Of IP Address
I am establishing a database for the purpose of logging access to my secure webserver and am wanting to make the database as efficient as I can because it will be doing a lot of work when the site goes live. What is the most efficient way in a MySQL table to store remote IP addresses? What data type should I use? Should I just go with a basic VARCHAR(15) to allow for 4 sets of 3 digits with 3 decimal separators, or is there a better way?
View Replies !
Efficient Placement Of Fields
is there any noticable efficiency is ordering the position of the field types?. Like say i place the join keys in a table at the end, and varchars and text at the top of the table does it really matter? Also when i do a query like select * from table where id IN (1), where 1 is usually a primary key int is that quicker than doing where id=1 or is there no difference and is it handling the int as a string or as an int?
View Replies !
Efficient Database Structure
I have been developing a new website and i need help in deciding the best database structure for it. The site is basically a dating website with various modules like blogs, videos, comments, friends, photos etc. I have created a member table that stores all the basic profile info and created seperate tables for friends, photos, messages, blogs etc and MemberID as foreign key. Now on profile page, i want to display all info related to member like his profile info, his photos, friend list, messages etc and i have to execute 7-10 short queries on profile page for this. Also, i think Joins will not be much helpful as there is one to many relationship e.g there can be more than 1 photo for a member and i am saving each photo in a seperate record. Similar is the case for other tables?
View Replies !
Creating An Efficient Database
i'm currently writing a web based catalogue system in php using a mysql database. the catalogue has a number of products in it from different brands. i would like to know if it is more efficient to have each brand in a separate table then a "master" table just listing the brand name and corresponding table or all the products in one large table and a "master" table listing each brand in the large table. the large product table would of course have a field to state which brand the product was from. the efficiency would be based on users being able to access the database via html browsers using php and also search the database.
View Replies !
How To Make Index Most Efficient
I have a large table (> 3,000,000 records). Each records contain a primary key like 'id' and a lots of attributes like 'age', 'department'. I want to build some indices to accelerate my query. I read the document which says that too many indices may slow down the INSERT and UPDATE operations. So is there any rule on how to set indices in such table? If I create an index for every field, would that be a very bad idea? If I create an index on each of two fields but not on their combination, will the indices contribute to queries on the combination?
View Replies !
Efficient Way To Count Rows
I'm trying to get the number of rows in a table with a very large number of records in it (~9 million). When I run a select count(*) for some criteria (where name='something', etc) it takes around 6-8 secs for the query to return the value. I tried by using SQL_CALC_FOUND_ROWS with a very small LIMIT but then the query was taking even longer. I'm using InnoDB, with query caching enabled. I could look at the information_schema and get the approximate row count but whenever I use a where clause it'll be way off mark.
View Replies !
Is This INSERT With SELECT As Efficient As Possible?
I'm writing a pretty complex web app and will be repeating many times over a query very similar to below and need to know if there is a more efficient way to do it. If anyone has input, I'd be happy to hear: INSERT INTO table (somecolumn) VALUES ((SELECT id FROM other_table WHERE foo = 'bar'))
View Replies !
Is This An Efficient MySQL Setup?
I would like to offload the MySQL server from my dedicated box in order to speed up page loads. I don't have an additional dedicated server so my only option is to get a VPS. But is offloading the MySQL server to a VPS, albeit a modest one, even worthwhile? I understand this is a very broad question because I am not providing any details but that being the case I am expecting a broad answer
View Replies !
Efficient Way To Left Join?
What is the most efficient way to do a table join, but even if there is no matching foreign key, still return the table on the left. SELECT col, col2, COUNT(col3) FROM tbl1 t1 LEFT JOIN tbl2 t2 ON t1.id = t2.id GROUP BY t1.id But this only works if there is a matching foreign key on t2, I would like all t1 rows to return regardless of whether they exist on t2 or not.
View Replies !
Most Efficient Way To Extract Limited Data
I am currently using the following code, is it the most efficient way to extract and sort the 6 items from the database. The database currently holds over 2,500,000 rows and I want to extract the data as efficiently and quickly as possible. Code:
View Replies !
Most Efficient, Stable Version Of MYSQL?
We're still using mysql 3.23.56, so it's time to upgrade! It has never once crashed on us, so we've been content with it so far. However, some of the inefficient query optimizations make us want to upgrade. Does anyone know what the current most efficient and stable version of MYSQL is? We don't require views or cursors or stored procedures or any of that... Just whatever's the most reliable and best optimized for fast query execution of fairly simple SQL. I know that 3.23 has some issues with using indices correctly during optimization (most notably in ORDER BY ... DESC). Is that problem fixed in 4?
View Replies !
Efficient Ways To Retrieve Specific Rows.
My site used forum software that I wrote myself (in Perl) which, up until now, has used flat files. This worked fine, however lately I've been wanting to do more stuff with user accounts, and had been eying MySQL for over a year. Finally I've decided to start off small by converting the forum's account system to a MySQL database (and convert the rest later after I'm comfortable). So far everything is working fine, and I've figured out how to create the table, insert records, modify records, and so on. However I had a question on what was the most efficient way to retrieve information about a user as I read through the flat file containing messages. As each message is read I want to find that user's relevant information, build it into my output, then continue on to the next message. Now here's my problem. Since I'm reading a flat file in a way that was once trying to be somewhat memory efficient (on really slow server way back when, trying to avoid arrays and hashes) I'm finding that I'm having to send separate questions to MySQL as each message comes up. Say from 1 to 20 very simple questions to complete printing the page. Also, sometimes the questions might be repetitive - since I'm not storing any of the results in memory, if a user appears twice I ask about it twice. Would I really be better off trying to find a way to consolidate all distinct users into a single question? Or is MySQL efficient enough that this isn't really much of a concern?
View Replies !
Efficient String Compare To A Group Of Values
(Post edited to change "WHERE field = options" to "WHERE field IN options"). I have a website using PHP and MySQL 5.0; the database holds personal names, addresses and various information. Right now I'm building a query which checks whether a person lives in a certain group of UK counties. I've got arrays set up in PHP which hold the groups of counties, and I've dynamically created a "group" from each array. So, in PHP, $county_array = array('Kent','Surrey','London','Essex','...'); $county_group gets set to "('kent','surrey','london','essex',...)" Each group may hold perhaps 20 individual counties. Now, for my database query, I use: $query = "SELECT ... WHERE lower(county) IN $county_group"; Is the "WHERE field IN (a|b|c|d|e|...)" a slow comparison in MySQL? Is there a more efficient way to do it? Is this something that stored procedures can help me with?
View Replies !
The Most Efficient Manner To Update Multiple Rows
I have a table that holds information about people The table is quite large (300,000 rows) I need to write code that update many of the rows in the table Here is what I need to do Two of the fields that I store about each person are: location and country The location is some free text which the user can type in, while the country is the ISO country code, like "US", "ES" and so on I need go over all the records in the database, and for those that have a non-empty location field, to extract the value and to try to guess what country the user is from and to update the Country field accordingly My problem is that I might find that most of the records need updating, and this might lead to 100,00+ update statements I cannot use LOAD DATA as this is product database Is there a way to update multiple rows (each with a different value and condition in a single query?
View Replies !
Efficient SELECT From Multiple Tables With Same Formatting?
Let's say I have 2 tables with the exact same formatting (field/column names, etc) -- the only difference is the name of the tables.. one is called "table1" the other is called "table2" If I wanted to query both of the together is this legal and good coding practice? "SELECT id, headline, permalink, body FROM table2007_11, table2007_12 WHERE id=23" understandly I get this error: Column 'id' in field list is ambiguous" is there a better way to do this than printing out each table name for each field individually? (ie. id. table2007_11, id. table2007_12, etc)?
View Replies !
Efficient Country/Province City Setup
I currently have a program that does a global query of all of the data. I want to break it down so as to drill the user down to thier local area. for example: Country Province Region City As you can see, a country can have many provinces, a province many regions and a region many cities or a province many cities. What is the most effecinet way to do this in MySql? I'm OK with doing single tables. ie: INSERT INTO Province VALUES (NULL, 'Ontario', 'ON'); but I'm unclear if that's the way it should be done. I'm using PHP on the client side.
View Replies !
Most Efficient Way Of Storing Data From Multiple Accounts
As part of a system I am putting together I need to allow users to create thier own accounts on my servers. Each user can create their own account, and then have their users register for it. Each account needs it data seperate from the others, a member registered for one account should not be able to view another account and a username registered with one account should still be available to the other accounts. The ways I have been looking at are: 1. Create a new database for every account created so that all users are kept in seperate databases. 2. Have one table for users, one for topics, one for posts etc and then associate each row within this table with the relevent account. So for example a user could register with the forum with the ID 4, so their user entry would be Userid: 234 Username: xxxx Password: xxxx Forumid: 4 Then when a new member registers with any account I simply check that there is not another user with the same account ID and username. Indexes on relevent fields in this system could help speed up huge tables. I expect to quickly have 20,000 plus accounts (and in theory it could go up to hundreds of thousands). I guess my question is which of these methods is better from a speed point of view once we get a large number of accounts and users. Also, are there restrictions on the number of fields in a table that could cause problems? Using MySQL by the way, on an Apache server.
View Replies !
Most Efficient Date Field Type For Selects?
I have searched this forum seeking the answer to this question, to no avail. I am making an events diary that will have a lot of entries. Entries will be entered in all kinds of order, including multiple events spanning a year or so. Even multiple events of the same type will be individual entries. (e.g. weekly chess club meeting every weds at 7pm will be 52 entries) My question is : Which is the best format to store dates? Native mysql date type Integer timestamp A.n.other? I suppose a worst case scenario is imagine I want to generate a calendar for July (31 days) and I want to show how many events there are for each day. 1st July (13 events) 2nd July (22 events) and so on ... it is possible that any given day could have up to 100 entries. What in your opinion is the best column type to use and ss there any kind of Indexing I can use to speed up select by dates? Or any other tip or trick you'd be prepared to share with me?
View Replies !
GIS Indexing..
I have been testing mysql's GIS support for quite some time now. I have one question. Suppose I execute this simple query: select * from node where MRBWithin(node.shape_geo, LINESTRING(0 0, 200 200)); This will select all node records that are bounded in in the ractangle (0,0), (0,200), (200,200) and (200,0). Is there anyway to get mysql to return the results in the order of 'closest to the center point of MBR'? Ie the center point (100,100). The result set would be something like: (101, 105) (90, 80) (20, 35) (199,173)
View Replies !
What Is Key Indexing
I hope this is not a "too complex" questions, so please help me with these: 1. What is key indexing (in MySQL)? 2. Assume I already have a database (and all tables inside), how do I perform a "key indexing" in my database? 3. And can you give me one simple sample of implementation?
View Replies !
Indexing FKs
Should I index fks seperately in the linking table (in relation n:m) (if I already indexed fks usign pk). heres an example: [student]-1---n-[student_lessons]-n----1-[lesson] CREATE TABLE students_lessons ( student_id INT(11) NOT NULL, lesson_id INT(11) NOT NULL, PRIMARY KEY (student_id, lesson_id) KEY fk1(student_id), /*add this??*/ KEY fk2(lesson_id) /*add this??*/ )"
View Replies !
Use Indexing
I will make an online game with max 5000 accaunts and max 20000 character. Online users will be always 200-300. I want to make it with mysql becouse if I use array and it crashes all the data will be lost. I am new in Mysql. I want to know how to use indexes (How to get the id of a character and how to get character data of an ID). There was many tutorials on http://dev.mysql.com/doc/ for indexes but I cant understand it! :S If I use fixed row length would it be faster? Can I get memorypointers to speed up indexing?
View Replies !
Indexing?
I have a Database Table consisting of 16 rows. One is a Card number row which is Unique. The database size is over 3lac. I want to Optimize the database. I read about indexing. How to go about this?? Do i do single indexing or double?
View Replies !
MySql DB Indexing
I'm a bit confused about indexing using MySQL.I have something like this: Table: PEOPLE Values: user_id first_name last_name email I put an index on user_id to speed up my search for: select first_name, last_name, email from PEOPLE where user_id=xxx; Once I set up the index, does it only hold true for the data that was currently in the table at the time the index was created? In other words, if I add 100 more users tomorrow,will I have to rebuild the index on user_id, or does MySQL automatically adjust for that?
View Replies !
Indexing Process
Is there a way to check the current status of the indexing process in MySQL [Ver 12.21 Distrib 4.0.15, for apple-darwin7.2.0 (powerpc)]? I am indexing a huge database and I would like to have an estimation of the end of the process. Is the indexing time linear or should I expect any special delay?
View Replies !
Fulltext And Indexing
I have a database with three text fields in it , each with a fulltext index. If I do a fulltext query against one of the fields, the explain says it will use the index. If I do a query on more than one of the fulltext fields, it doesn't use any of the indexes. What am I missing here?
View Replies !
Fulltext Indexing
If I have a query like: Quote: SELECT ided, merch.mid, mer, cost, ran, name, imgurl, brands FROM merch, products WHERE products.mid=merch.mid AND mcat='$cat' AND group=1 AND cost BETWEEN '$min' AND '$max' AND approved=1 AND MATCH (brands, cat) AGAINST ('$keyword' IN BOOLEAN MODE) How would I set up the index keys. I have done a fulltext index key on "brands, cat" Is this all I have to do or will I need to also do an index on "mcat, group, cost" so that it speeds up the query. I am doing this for optimization and want my queries to be as fast as possible. Take in mind that approved is from the merch table and all the columns from both of the indexes I mentioned above are from the products table.
View Replies !
Using B-tree Indexing
If I use a b-tree index on a memory table, then the index will work with column comparisons such as <= and >= and <>? I am just trying to make sure, because memory with hash indexing, only make use of the index when using = and <=>.
View Replies !
Indexing Two Columns
I have a table with two columns, x and Y. Could someone please elucidate the difference between CREATE INDEX joint_index ON mytable (x,y); and CREATE INDEX index1 ON mytable (x); CREATE INDEX index2 ON mytable (y); as far as a reflexive join of mytable - e.g., '..on a.x = b.x and a.y < b.y', would be concerned - or just point to a good online reference on indexes?
View Replies !
Frequent Indexing
My table has ~ 2 million records in it. Out of those records, only a relatively small set (~200) are queried and updated frequently. The rest are queried but almost never updated. Furthermore, I can reliably predict which of the records are going to be frequently updated and which ones are not. The issue is that even though only the same 200 records are being updated, the indexes have to be adjusted for all 2 million of them. To speed things up, I'm considering two options: (1) Split the table into two and have my web app switch between them as necessary. (2) Add a frequently_updated field to my table and index based on that. I'd like to use option 2 since it's a lot less likely to break something in the app, but I'm not sure how it compares performance-wise. Let's say for option 2, all indexes are of the form [frequently_updated, other_field(s)]. How then is the index structured? Specifically, if I add or update a record where frequently_updated = true, is only a small part of the index updated (e.g. only one branch on a tree) or is the entire index updated?
View Replies !
Scheduling Indexing?
I've been using mysql for a while, but to be honest I've never stored a load of data into it. Untill now. Now my project is taking off and I'm adding about 50,000 rows a month. i.e. 70 or so an hour.
View Replies !
Auto Indexing
Are primary and foreign keys auto-indexed for MyISAM and InnoDB? Mysql reference manual says “Most MySQL indexes (PRIMARY KEY, UNIQUE, INDEX, and FULLTEXT) are stored in B-trees. Exceptions are that indexes on spatial data types use R-trees, and that MEMORY tables also support hash indexes.†If I understand correctly, primary key is auto-indexed but what's about foreign key? Actually I need to add indexes to my tables; if mysql internally maintains for both key types, I will not insert additional/same indexes.
View Replies !
Database Indexing
I have set up a very large MySQL database, with over 180MB of data. This data is read by my PHP script and converted into an XML file. These XML files tend to be between 25 and 35KB once generated. My knowledge of databases is average. I've taken one relational databases course, but not the "advanced" one, so I need help indexing them for best performance. Hopefully someone can help. (If not, hey, it was worth a shot). Here is the structure for four of the tables (5 total tables, but the other one isnt currently in use). Code:
View Replies !
Indexing MySQL?
I keep on hearing alot about indexing mysql but can't seem to figure out how to do it, I'm running mysql on linux redhat 7.2 could some one explain to me how to do this as i'm not very familar with mysql administration tasks.
View Replies !
MySQL Indexing
Had much experience with indexing tables for optimality, but the product-search on a site I've been involved with is painfully slow, and I want to speed it up. It's based on OSCommerce, which is woefully inefficient, but there's nothing to be done about that, so will have to struggle with the database. The particulatly noticeable slowness is (ironically enough...) with the 'quick search' functionality. The query is essentially: SELECT p.image, pd.name, pd.short_description, p.price FROM products p JOIN products_description pd ON (p.id=pd.productid) WHERE pd.lang=1 AND pd.name LIKE '%keyword%' OR pd.description LIKE '%keyword%' OR p.model LIKE '%keyword%'; It's annoying that the system uses two tables when (in this case at least, with only one language) one would suffice. When in the context of the page, a search using the query as above takes about 15-20 seconds to load (this is running on a more than adequate server, on a local network), running just the query from the MySQL console registers about 7 seconds. What should I be indexing? If it were ANDs joining the WHERE-conditions then it'd be simple, but is there anything that can be done? The OSC paging-system does a SELECT COUNT(*) FROM... using the same query as above, and this is rather slow as well, so any pointers in that direction would be appreciated. I'm more familiar with postgres, and so don't have much of a clue about FULLTEXT indexing? Would this be of any benefit? The databases contains around 50k products and accompanying descriptions so while not tiny, is not exactly huge and shouldn't be this slow.
View Replies !
Indexing For Sorts
should you make a column that will always be sorted into an index? for example: a column that counts the number of views to a particular item. (in this case, the most common usage will be to fetch the item that is most viewed... my method SELECT * FROM counter SORT BY count DESC, LIMIT 1
View Replies !
Indexing Question
I know this is probably very simple but I'm lost. I want to give my MySql database an index so that the first item has the value 0, the second has the value 1 and so on. I assumed I did this with a primary key set to auto increment but when I delete a record it leaves a gap in the index (if I have files with a primary key of 0, 1, 2, 3 and delete record 2 my index looks like 0, 1, 3, rather than updating the numbers to 0, 1, 2). How do I set such an index up? Do I even need to do this or is this built into mysql in some way? If so, what do I need to refer to if I am, for example, deleting a record (what do I need to replace the '?' below with to delete my second record?): $delete = "DELETE FROM Articles WHERE ?=1"; mysql_query($delete); I'm using php and phpMyAdmin to communicate with the database.
View Replies !
|
TRACKER
