Efficient Ways To Retrieve Specific Rows.
My site used forum software that I wrote myself (in Perl) which, up until
now, has used flat files. This worked fine, however lately I've been wanting
to do more stuff with user accounts, and had been eying MySQL for over a
year.
Finally I've decided to start off small by converting the forum's account
system to a MySQL database (and convert the rest later after I'm
comfortable). So far everything is working fine, and I've figured out how to
create the table, insert records, modify records, and so on.
However I had a question on what was the most efficient way to retrieve
information about a user as I read through the flat file containing
messages.
As each message is read I want to find that user's relevant information,
build it into my output, then continue on to the next message. Now here's my
problem. Since I'm reading a flat file in a way that was once trying to be
somewhat memory efficient (on really slow server way back when, trying to
avoid arrays and hashes) I'm finding that I'm having to send separate
questions to MySQL as each message comes up. Say from 1 to 20 very simple
questions to complete printing the page. Also, sometimes the questions might
be repetitive - since I'm not storing any of the results in memory, if a
user appears twice I ask about it twice.
Would I really be better off trying to find a way to consolidate all
distinct users into a single question? Or is MySQL efficient enough that
this isn't really much of a concern?
View Complete Forum Thread with Replies
Related Forum Messages:
Efficient Way To Count Rows
I'm trying to get the number of rows in a table with a very large number of records in it (~9 million). When I run a select count(*) for some criteria (where name='something', etc) it takes around 6-8 secs for the query to return the value. I tried by using SQL_CALC_FOUND_ROWS with a very small LIMIT but then the query was taking even longer. I'm using InnoDB, with query caching enabled. I could look at the information_schema and get the approximate row count but whenever I use a where clause it'll be way off mark.
View Replies !
The Most Efficient Manner To Update Multiple Rows
I have a table that holds information about people The table is quite large (300,000 rows) I need to write code that update many of the rows in the table Here is what I need to do Two of the fields that I store about each person are: location and country The location is some free text which the user can type in, while the country is the ISO country code, like "US", "ES" and so on I need go over all the records in the database, and for those that have a non-empty location field, to extract the value and to try to guess what country the user is from and to update the Country field accordingly My problem is that I might find that most of the records need updating, and this might lead to 100,00+ update statements I cannot use LOAD DATA as this is product database Is there a way to update multiple rows (each with a different value and condition in a single query?
View Replies !
Retrieve Most Recent Rows
I have a simple table with four columns. ID (auto_increment), username, score, and date (unix timestamp). How could I, if possible, get the past 10 scores of each unique username? So if there were 15 different usernames, I would get 150 rows returned (assuming all of them had played 10+ games). Can this be done in a single query or do I need to run a query for each username?
View Replies !
Retrieve Number Of Rows
I've got a large table with a lot of entries. only two fields, one's the key (which auto incs) and the other's a text field...if I want to just retrieve the number of rows in the table, what's the best way to do this?I know I could tell mysql to sort the table by the key row and then take the first one, but wouldn't this make it analyze the entire table (which would waste resources, especially considering that this table will have thousands and thousands of entries)? what's a better way to do it?
View Replies !
Specific Rows
When I was creating a database I misassigned some rows and now I just want to change those 4 rows but can't figure out how to do it. I tried using this command Code: update variables set bl_id=16 where bl_id in (SELECT bl_id from variables where var_id between 379 and 382)
View Replies !
Retrieve Everything AND Count Rows In One Query
set rsminmax = con.execute("SELECT * FROM `minirules_minmax` where RuleId = '" & contractId & "'") set rsminmax2 = con.execute("SELECT COUNT(*) FROM `minirules_minmax` where RuleId = '" & contractId & "'") Is there any way to do this in one SQL query?
View Replies !
Only Selecting Specific Rows
Fields: studentID, reviewID, date I want to get all rows in a certain date range where there are at least 3 rows for a given studentID. In other words... Case 1: student has written 2 or fewer reviews during the date range. Don't retrieve these rows. Case 2: student has written 3 or more reviews during the date range. Retrieve ALL of their reviews during that time. So if they wrote 5 reviews, the query will have 5 rows for their studentID. Any ideas for this query? So far I got the date range to work using this. I'm struggling with the 3+ reviews part. SELECT `studentID`, `reviewID`, `date` FROM `review` WHERE `date` BETWEEN '2007-10-21' AND '2007-11-09' ORDER by studentID, `date` Any ideas?
View Replies !
Duplicate Specific Set Of Rows
I'm receiving a query error and im not sure why. It is supposed to duplicate a specific set of rows in a table. Which rows is determined by a different table. Is therea clear syntax error? MySQL INSERT INTO listings VALUES (SELECT * FROM listings WHERE id = (SELECT list_id FROM groupings WHERE group_id IN(2) ) ) The MySQL error said it was Quote: near ''(SELECT * FROM listings WHERE id = '(SELECT list_id FROM g' e39m5
View Replies !
How To Retrieve The Total Number Of Rows When Using LIMIT X, XXX
With php+mysql, $results -> num_rows returns the number of rows of the current query, however with LIMIT X, XXX in that query the $results -> num_rows only contains the LIMITED number of rows rather than the total amount of rows meeting the WHERE clause without LIMIT. e.g. With LIMIT 0, 100 in my query $results -> num_rows returns 100 rather than 2000 which is the total number of rows had I not limited the returning results for paging. Some search site has the capability of paging big amount of results while displaying also the total amount. Does it require 2 or more queries to do this? Is it possible to do it with just 1 query? So what's the most efficient way to achieve this: Quote: Searching results 1 - 100, 2,653 total ... ...
View Replies !
Retrieve Two Rows From Table And Show Result In Single Row
I have three tables, one table is with personal details, another table is with contact details and another table with contact type 1. personal_details 2. contact_details 3. contact_type the structure of the tables are given below personal_details ----------------- id | first_name | last_name 1 john | smith 2 Kevin | McDonald contact_type ------------ id | type 1 | home 2 | work contact_details ---------------- id | personal_details_id | contact_type_id | address | zip | state | phone 1 | 1 | 1 | abc | 123 | CA | 1234 2 | 1 | 2 | pqr | 321 | TX | 4321 3 | 2 | 1 | xyz | 456 | WA | 9876 4 | 2 | 2 | asdf | 654 | FL | 6789 in the contact_details, we have two rows for one person. first row is his home address and second row is his work address. i need to select both the home address and work address from the table and show in all these details in the result... the structure of my result has to be as given below first_name | last_name | home_address | home_zip | home_state | home_phone | work_addres | work_zip | work_state | work_phone
View Replies !
MYSQL: Retrieve One Row With Data From JOINED Tables With Multiple Matching Rows
I'm not sure this is possible in one SQL query: I have a table called TB_EMAIL and another called TB_OTHER_BLOB. TB_EMAIL contains details like: to, from, subject, body and since I will be reusing headers and footers I aslo store header and footer reference to TB_OTHER_BLOB in TB_EMAIL. What I need is a row (given its id) with all columns (except the references to TB_OTHER_BLOB) from TB_EMAIL and two extra columns one being email_header and the other email_footer which are in two different rows in TB_OTHER_BLOB What I came up with is: Expand|Select|Wrap|Line Numbers
View Replies !
Ways Around Using SELECT *
What ways are they around using SELECT * on a page that loads in all fields from various tables? I can't use SELECT 1,2,3,4 because one table may not have 3,4 etc. Is there any efficient way to get all the field names in a table and then assign them to a variable other than use a SELECT * then $fields = mysql_field_name($result, $i) to list all the fields and then doing my SELECT $fields? I can't really see that offering a performance gain although I may be wrong as every time the database is queried it should use the $fields variable rather than SELECT *.
View Replies !
Replication / Synchronization Both Ways
I have this situation. A central mysql database is located somewhere on a hosting partner's space. I've developed a desktop application to communicate with that mysql database. The users should be able to work anywhere, so the desktop application always connects to that online mysql database. But as the database grows, I'm suffering some performance problems. Therefore I would like to introduce some kind of replication so that all users can work locally and the central DB is always synchronized when a user changes something. I've read about replication and it says that there'll always be a master and a slave DB, but I want actually both ways to be "master"... I mean : when a user starts up the program it should connect to the online DB to sync the local DB so that it's exactly the same and the user can start working normally. But when he closes down the tool, the same has to happen but in the other direction, so that the central DB is aware of the changes the user has done. To avoid problems this sync has to happen like every 5 minutes (the policy will be : first update the central DB with the changes made locally and then the other way around). How can I achieve this both-way replication?
View Replies !
What's The Most Efficient Way?
I've done a database scheme and wonder what the most efficient way of storing certain entries would be. The site will have news, reviews and tutorials and these will be under the same categories. So I'm wondering what the best way to do design the database scheme would be. Have one table like I've done now in the "content" table with the posibilty to differentiate the entries with the "post_type" field or have three different tables? Below is the table "content". Field Type Null Default post_id int(10) No post_date datetime No 0000-00-00 00:00:00 post_text text No post_title text No post_cat_id int(4) No 0 post_description text No post_name varchar(200) No post_type int(4) No 0
View Replies !
How Efficient Is In() , And How Much Is The Most That You Should Put In An In()?
I am just wondering as to the efficiency of the in() function in MySQL. select field1,field2,...,fieldn from table where id in(1,2,3,4,5,6, ... , n); assuming: id is an indexed field. table is a VERY big table (100 000+ - 1 million+ records) what do you think is the largest number of values you could pass to the in() function without completly flattening your server?
View Replies !
Most Efficient YES/NO...
I) Table_A has fields Data_1, Data_2, Data_3, and Data_4; I need to determine whether value 'X' is present (at least once) in any of the Data_N fields. All I need is a YES/NO answer: YES, 'X' is present (at least once) in one of the Data_N fields of Table_A [Record number(s) not required!], or NO, 'X' is not present in any of the Data_N fields in any of the records in Table_A. II) Similar scenario, but this time Table_A has fields RecID, Data_1, Data_2, Data_3, and Data_4. I need to determine the RecID (none, one, or more) of every record that has value 'X' in any of the Data_N fields. I'm looking for the query that will be fastest and most elegant to implement.
View Replies !
Quickest And Most Efficient
i run an online game. I want to give each user a ranking, based on how high thier score is.If for example, i had 20,000 players, i dont want to have to update them all one by one, it may strain the server and take a long time. Is there another way i can assign a rank number (rank 1 has the top score and so on)
View Replies !
Efficient UPDATE
I have a table with the following structure; CREATE TABLE my_table ( id_1 int(11) NOT NULL , id_2 int(10) NOT NULL , stauts tinyint(1) NOT NULL DEFAULT 0 , PRIMARY KEY (id_1) ) Engine =InnoDB'; The table currently has arround 100,000 entries. When I try to run variations of the following statement it is taking around 4 seconds per query; UPDATE IGNORE my_table SET id_1 = 74240, id_2 = 5 I need it to be running a lot faster than 4 seconds per query as I need to update upwards of 100,000 records a day! My server is fairly beefy, a 3 gig dual core opeteron and is generaly running below 1.0 load.
View Replies !
Efficient Tables.
I am starting out on a project where I need to store GPS information. The data consists largely of a series of "Points" each consisting of a longitude,latitude and elevation. On a typical "route" there could be hundreds of points. My question is how can I efficiently store this information. It does not sound sensible to normalise this and add hundreds of rows to a table for each "route". Sample data is along the lines of : - <trkpt lat="54.016942977905273" lon="-1.4903640747070313"> <ele>82.330078125</ele> <time>2006-09-03T07:35:41Z</time> </trkpt> - <trkpt lat="54.016938870772719" lon="-1.490332055836916"> <ele>0</ele> <time>2006-09-03T07:35:42Z</time> </trkpt>
View Replies !
Efficient Storage Of IP Address
I am establishing a database for the purpose of logging access to my secure webserver and am wanting to make the database as efficient as I can because it will be doing a lot of work when the site goes live. What is the most efficient way in a MySQL table to store remote IP addresses? What data type should I use? Should I just go with a basic VARCHAR(15) to allow for 4 sets of 3 digits with 3 decimal separators, or is there a better way?
View Replies !
Efficient Placement Of Fields
is there any noticable efficiency is ordering the position of the field types?. Like say i place the join keys in a table at the end, and varchars and text at the top of the table does it really matter? Also when i do a query like select * from table where id IN (1), where 1 is usually a primary key int is that quicker than doing where id=1 or is there no difference and is it handling the int as a string or as an int?
View Replies !
Efficient Database Structure
I have been developing a new website and i need help in deciding the best database structure for it. The site is basically a dating website with various modules like blogs, videos, comments, friends, photos etc. I have created a member table that stores all the basic profile info and created seperate tables for friends, photos, messages, blogs etc and MemberID as foreign key. Now on profile page, i want to display all info related to member like his profile info, his photos, friend list, messages etc and i have to execute 7-10 short queries on profile page for this. Also, i think Joins will not be much helpful as there is one to many relationship e.g there can be more than 1 photo for a member and i am saving each photo in a seperate record. Similar is the case for other tables?
View Replies !
Creating An Efficient Database
i'm currently writing a web based catalogue system in php using a mysql database. the catalogue has a number of products in it from different brands. i would like to know if it is more efficient to have each brand in a separate table then a "master" table just listing the brand name and corresponding table or all the products in one large table and a "master" table listing each brand in the large table. the large product table would of course have a field to state which brand the product was from. the efficiency would be based on users being able to access the database via html browsers using php and also search the database.
View Replies !
How To Make Index Most Efficient
I have a large table (> 3,000,000 records). Each records contain a primary key like 'id' and a lots of attributes like 'age', 'department'. I want to build some indices to accelerate my query. I read the document which says that too many indices may slow down the INSERT and UPDATE operations. So is there any rule on how to set indices in such table? If I create an index for every field, would that be a very bad idea? If I create an index on each of two fields but not on their combination, will the indices contribute to queries on the combination?
View Replies !
Efficient Way Of Mass Indexing?
I have been working on a program that will populate and index a database. The populating doesn't take too long, but the indexing does. My question is: Is there a better approach to indexing this table than the one I'm using right now? I'm doing this through QtSql. headers is a QString array of 48 column names I want to index. for (int i = 0; i < headers.size(); ++i) { q.exec("ALTER TABLE mysqltable ADD INDEX(" + headers[i] + ");"); } example run for 20000 rows: viper,david $ time ./main -r 20000 Database Connection Established 0.520u 0.060s 0:21.27 2.7% 0+0k 0+0io 0pf+0w
View Replies !
Is This INSERT With SELECT As Efficient As Possible?
I'm writing a pretty complex web app and will be repeating many times over a query very similar to below and need to know if there is a more efficient way to do it. If anyone has input, I'd be happy to hear: INSERT INTO table (somecolumn) VALUES ((SELECT id FROM other_table WHERE foo = 'bar'))
View Replies !
Is This An Efficient MySQL Setup?
I would like to offload the MySQL server from my dedicated box in order to speed up page loads. I don't have an additional dedicated server so my only option is to get a VPS. But is offloading the MySQL server to a VPS, albeit a modest one, even worthwhile? I understand this is a very broad question because I am not providing any details but that being the case I am expecting a broad answer
View Replies !
Efficient Way To Left Join?
What is the most efficient way to do a table join, but even if there is no matching foreign key, still return the table on the left. SELECT col, col2, COUNT(col3) FROM tbl1 t1 LEFT JOIN tbl2 t2 ON t1.id = t2.id GROUP BY t1.id But this only works if there is a matching foreign key on t2, I would like all t1 rows to return regardless of whether they exist on t2 or not.
View Replies !
Most Efficient Way To Extract Limited Data
I am currently using the following code, is it the most efficient way to extract and sort the 6 items from the database. The database currently holds over 2,500,000 rows and I want to extract the data as efficiently and quickly as possible. Code:
View Replies !
Most Efficient, Stable Version Of MYSQL?
We're still using mysql 3.23.56, so it's time to upgrade! It has never once crashed on us, so we've been content with it so far. However, some of the inefficient query optimizations make us want to upgrade. Does anyone know what the current most efficient and stable version of MYSQL is? We don't require views or cursors or stored procedures or any of that... Just whatever's the most reliable and best optimized for fast query execution of fairly simple SQL. I know that 3.23 has some issues with using indices correctly during optimization (most notably in ORDER BY ... DESC). Is that problem fixed in 4?
View Replies !
Efficient String Compare To A Group Of Values
(Post edited to change "WHERE field = options" to "WHERE field IN options"). I have a website using PHP and MySQL 5.0; the database holds personal names, addresses and various information. Right now I'm building a query which checks whether a person lives in a certain group of UK counties. I've got arrays set up in PHP which hold the groups of counties, and I've dynamically created a "group" from each array. So, in PHP, $county_array = array('Kent','Surrey','London','Essex','...'); $county_group gets set to "('kent','surrey','london','essex',...)" Each group may hold perhaps 20 individual counties. Now, for my database query, I use: $query = "SELECT ... WHERE lower(county) IN $county_group"; Is the "WHERE field IN (a|b|c|d|e|...)" a slow comparison in MySQL? Is there a more efficient way to do it? Is this something that stored procedures can help me with?
View Replies !
Efficient SELECT From Multiple Tables With Same Formatting?
Let's say I have 2 tables with the exact same formatting (field/column names, etc) -- the only difference is the name of the tables.. one is called "table1" the other is called "table2" If I wanted to query both of the together is this legal and good coding practice? "SELECT id, headline, permalink, body FROM table2007_11, table2007_12 WHERE id=23" understandly I get this error: Column 'id' in field list is ambiguous" is there a better way to do this than printing out each table name for each field individually? (ie. id. table2007_11, id. table2007_12, etc)?
View Replies !
Efficient Country/Province City Setup
I currently have a program that does a global query of all of the data. I want to break it down so as to drill the user down to thier local area. for example: Country Province Region City As you can see, a country can have many provinces, a province many regions and a region many cities or a province many cities. What is the most effecinet way to do this in MySql? I'm OK with doing single tables. ie: INSERT INTO Province VALUES (NULL, 'Ontario', 'ON'); but I'm unclear if that's the way it should be done. I'm using PHP on the client side.
View Replies !
Most Efficient Way Of Storing Data From Multiple Accounts
As part of a system I am putting together I need to allow users to create thier own accounts on my servers. Each user can create their own account, and then have their users register for it. Each account needs it data seperate from the others, a member registered for one account should not be able to view another account and a username registered with one account should still be available to the other accounts. The ways I have been looking at are: 1. Create a new database for every account created so that all users are kept in seperate databases. 2. Have one table for users, one for topics, one for posts etc and then associate each row within this table with the relevent account. So for example a user could register with the forum with the ID 4, so their user entry would be Userid: 234 Username: xxxx Password: xxxx Forumid: 4 Then when a new member registers with any account I simply check that there is not another user with the same account ID and username. Indexes on relevent fields in this system could help speed up huge tables. I expect to quickly have 20,000 plus accounts (and in theory it could go up to hundreds of thousands). I guess my question is which of these methods is better from a speed point of view once we get a large number of accounts and users. Also, are there restrictions on the number of fields in a table that could cause problems? Using MySQL by the way, on an Apache server.
View Replies !
Most Efficient Date Field Type For Selects?
I have searched this forum seeking the answer to this question, to no avail. I am making an events diary that will have a lot of entries. Entries will be entered in all kinds of order, including multiple events spanning a year or so. Even multiple events of the same type will be individual entries. (e.g. weekly chess club meeting every weds at 7pm will be 52 entries) My question is : Which is the best format to store dates? Native mysql date type Integer timestamp A.n.other? I suppose a worst case scenario is imagine I want to generate a calendar for July (31 days) and I want to show how many events there are for each day. 1st July (13 events) 2nd July (22 events) and so on ... it is possible that any given day could have up to 100 entries. What in your opinion is the best column type to use and ss there any kind of Indexing I can use to speed up select by dates? Or any other tip or trick you'd be prepared to share with me?
View Replies !
Retrieve Primary Key
I am doing a survey with many pages. Everyone is given the same password to get into the survey (bulk mailing) but since it is to be done anonymously, they have no unique id assigned to them. My problem is that I don't know how to save the data without it (and then select it again). I usually have the following (passcode is the unique number usually assigned to them): .....
View Replies !
Retrieve Images
I was wondering if is it possible for me to store images in mysql n den retrieve it out to thru the sql Select statement? If so how wld i go abt writing mysql Statement?? Pls can anyone guide me here im rather new in it.
View Replies !
|
TRACKER
