<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Tech Stuff &#187; mysql</title>
	<atom:link href="http://tech-stuff.org/tag/mysql/feed/" rel="self" type="application/rss+xml" />
	<link>http://tech-stuff.org</link>
	<description>Hosting Tips &#38; Tricks</description>
	<lastBuildDate>Fri, 20 May 2011 14:08:32 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Optimize mysql server/queries</title>
		<link>http://tech-stuff.org/optimize-mysql-serverqueries/</link>
		<comments>http://tech-stuff.org/optimize-mysql-serverqueries/#comments</comments>
		<pubDate>Wed, 09 Feb 2011 07:25:20 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[hosting]]></category>
		<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[optimization]]></category>

		<guid isPermaLink="false">http://tech-stuff.org/?p=180</guid>
		<description><![CDATA[On a mysql server there are a lot of queries to optimize and a lot of load generated by them. I&#8217;ll try to present the most usual optimization issues and how to identify them. First you&#8217;ll have to check mysql service settings . You can check them manually by following commands in mysql command line: [...]]]></description>
			<content:encoded><![CDATA[<p>On a mysql server there are a lot of queries to optimize and a lot of load generated by them. I&#8217;ll try to present the most usual optimization issues and how to identify them.</p>
<p>First you&#8217;ll have to check mysql service settings . You can check them manually by following commands in mysql command line:</p>
<p>mysql&gt;show variables;</p>
<p>or</p>
<p>mysql&gt;show variables like &#8216;%cache%&#8217;;</p>
<p>and</p>
<p>mysql&gt; show status;</p>
<p>you can check the counters and increase or decrease them according to their usage and limits.</p>
<p>The more easy way is to use some scripts to check mysql settings like  : MySQL performance tuning primer script (tuning-primer.sh).</p>
<p>First and most important optimization is to activate and set query_cache and query_cache_size to lower disk IO usage.</p>
<p>After that you will have to check the running queries . For that you&#8217;ll need to enable first :</p>
<p>log-slow-queries= /var/lib/mysql/mysql-slow.log</p>
<p>long-query-time=3</p>
<p>in my.cnf then touch /var/lib/mysql/mysql-slow.log and set owner as mysql user/group. You can optionally add &#8220;log-queries-not-using-indexes&#8221;</p>
<p>Open then /var/lib/mysql/mysql-slow.log and check slow queries one by one.</p>
<p>Not every query in the slow query logs is necessary a bad one. Look for queries where some of the following criteria are met:</p>
<p>A) &#8220;Rows_examined&#8221; is more than 2000</p>
<p>B) &#8220;Rows_examined&#8221; is less than 2000 but that query is being run 20 times a second.</p>
<p>C) &#8220;Rows_examined&#8221; is three times bigger than &#8220;Rows_sent&#8221;</p>
<p>(Note that these are rough criteria. Your mileage may vary depending upon your situation.)</p>
<p>Focus on the two or three worst queries at first. Once you&#8217;ve found a few, utilize the EXPLAIN statement to find a better way to restructure your query. See this link for an explanation and walkthrough of the EXPLAIN statement.</p>
<p>Now lets grep in the log file:</p>
<p>grep Rows_examined /var/lib/mysql/mysql-slow.log.old | sort -g -k9 -r |head -5</p>
<p>a sample result would be:</p>
<p># Query_time: 19  Lock_time: 0  Rows_sent: 31  Rows_examined: 6424367</p>
<p># Query_time: 58  Lock_time: 0  Rows_sent: 3886219  Rows_examined: 3886219</p>
<p># Query_time: 47  Lock_time: 0  Rows_sent: 3886219  Rows_examined: 3886219</p>
<p># Query_time: 40  Lock_time: 0  Rows_sent: 3886219  Rows_examined: 3886219</p>
<p># Query_time: 39  Lock_time: 0  Rows_sent: 3886219  Rows_examined: 3886219</p>
<p># search for the worst offender, here&#8217;s one way to do that:</p>
<p>grep -A 2 -B 2 3886219  /var/lib/mysql/mysql-slow.log</p>
<p># Time: 030611  18:49:05</p>
<p># User@Host: dbusername[dbusername] @ composer.com [166.233.115.222]</p>
<p># Query_time: 1  Lock_time: 0  Rows_sent: 3886219  Rows_examined: 3886219</p>
<p>SELECT msgs.*, username AS sender_username FROM msgs INNER JOIN users ON (users.id = msgs.sender_id) WHERE user_id=939 AND msgs.status != 1 AND del != 2 ORDER BY date DESC;</p>
<p># Open up a mysql shell to fix the problem:</p>
<p>[localhost]$ mysql -h mysql.exampledomain.com -u dbusername -pYOURPASSWORDHERENOSPACES dbname</p>
<p># EXPLAIN statement to show you how bad the query is.</p>
<p># Notice that 42000 rows rows of data are examined.</p>
<p>mysql&gt; EXPLAIN SELECT msgs.*, username AS sender_username FROM msgs INNER JOIN users ON (users.id = msgs.sender_id) WHERE user_id=939 AND msgs.status != 1 AND del != 2 ORDER BY date DESC;</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>| table | type   | possible_keys | key     | key_len | ref            | rows  | Extra                      |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>| msgs  | ALL    | NULL          | NULL    |    NULL | NULL           | 3886219 | where used; Using filesort |</p>
<p>| users | eq_ref | PRIMARY       | PRIMARY |       8 | msgs.sender_id |     1 |                            |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>2 rows in set (0.00 sec)</p>
<p># How long does the query take before fixing?  About 1 second.</p>
<p># (slightly modified for demostration purposes, but same result still).</p>
<p>mysql&gt; SELECT count(*) FROM msgs INNER JOIN users ON (users.id = msgs.sender_id) WHERE user_id=939 AND msgs.status != 1 AND del != 2 ORDER BY date DESC;</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>| count(*) |</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>|      631 |</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>1 row in set (1.03 sec)</p>
<p># You want to be indexing on stuff in your WHERE and JOIN statements.</p>
<p># specifically, where there is lots of uniqueness or &#8220;cardinality&#8221;.</p>
<p># user_id from above is really good, because there are lots of unique values</p>
<p># for user_id.  Same thing goes for users.id and msgs.sender_id</p>
<p># msgs.status won&#8217;t help that much (but won&#8217;t hurt) because mostly its values are</p>
<p># 0 and 1.  same thing goes for &#8220;del&#8221;.</p>
<p># Add an index on the user_id, and msgs.sender_id columns.</p>
<p># users.id is already indexed</p>
<p># Note: always try to add a key of length 10 first, it&#8217;s better (if possible).</p>
<p>mysql&gt; create index user_id_index on msgs(user_id(10));</p>
<p>ERROR 1089: Incorrect sub part key. The used key part isn&#8217;t a string, the used length is longer than the key part or the table handler doesn&#8217;t support unique sub keys</p>
<p>mysql&gt; create index user_id_index on msgs(user_id);</p>
<p>Query OK, 42857 rows affected (1.59 sec)</p>
<p>Records: 42857  Duplicates: 0  Warnings: 0</p>
<p>mysql&gt; create index sender_id_index on msgs(sender_id(10));ERROR 1089: Incorrect sub part key. The used key part isn&#8217;t a string, the used length is longer than the key part or the table handler doesn&#8217;t support unique sub keys</p>
<p>mysql&gt; create index sender_id_index on msgs(sender_id);</p>
<p>Query OK, 42858 rows affected (1.16 sec)</p>
<p>Records: 42858  Duplicates: 0  Warnings: 0</p>
<p># Check the indices, see if they look good.</p>
<p># They do look good.  Notice the high cardinatlity (uniqueness) of all three keys.</p>
<p>mysql&gt; show index from msgs;</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;+</p>
<p>| Table | Non_unique | Key_name        | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Comment |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;+</p>
<p>| msgs  |          0 | PRIMARY         |            1 | id          | A         |       42855 |     NULL | NULL   |         |</p>
<p>| msgs  |          1 | user_id_index   |            1 | user_id     | A         |        1224 |     NULL | NULL   |         |</p>
<p>| msgs  |          1 | sender_id_index |            1 | sender_id   | A         |        1071 |     NULL | NULL   |         |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;+</p>
<p>3 rows in set (0.00 sec)</p>
<p># Very good.</p>
<p># Now, check to see if your index actually improved anything.</p>
<p># First, check with the EXPLAIN statement.  Much better!</p>
<p>mysql&gt; EXPLAIN SELECT msgs.*, username AS sender_username FROM msgs INNER JOIN users ON (users.id = msgs.sender_id) WHERE user_id=939 AND msgs.status != 1 AND del != 2 ORDER BY date DESC;</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>| table | type   | possible_keys                 | key           | key_len | ref            | rows | Extra                      |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>| msgs  | ref    | user_id_index,sender_id_index | user_id_index |       8 | const          |  635 | where used; Using filesort |</p>
<p>| users | eq_ref | PRIMARY                       | PRIMARY       |       8 | msgs.sender_id |    1 |                            |</p>
<p>+&#8212;&#8212;-+&#8212;&#8212;&#8211;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;-+&#8212;&#8212;+&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;-+</p>
<p>2 rows in set (0.00 sec)</p>
<p>mysql&gt;</p>
<p># Now check the time it takes the query to complete.</p>
<p># Only 0.01 seconds to complete.  Much faster.</p>
<p>mysql&gt; SELECT count(*) FROM msgs INNER JOIN users ON (users.id = msgs.sender_id) WHERE user_id=939 AND msgs.status != 1 AND del != 2 ORDER BY date DESC;</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>| count(*) |</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>|      631 |</p>
<p>+&#8212;&#8212;&#8212;-+</p>
<p>1 row in set (0.01 sec)</p>
<p># Now start watching tail -f /var/lib/mysql/mysql-slow.log</p>
<p># to find out more tables that should be indexed</p>
<p>tail -f /var/lib/mysql/mysql-slow.log</p>
]]></content:encoded>
			<wfw:commentRss>http://tech-stuff.org/optimize-mysql-serverqueries/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>MySQL Tip: MySQL Server Has Gone Away Or Lost connection to server during query</title>
		<link>http://tech-stuff.org/mysql-tip-mysql-server-has-gone-away-or-lost-connection-to-server-during-query/</link>
		<comments>http://tech-stuff.org/mysql-tip-mysql-server-has-gone-away-or-lost-connection-to-server-during-query/#comments</comments>
		<pubDate>Wed, 19 Aug 2009 17:45:50 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[hosting]]></category>
		<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[mysql]]></category>

		<guid isPermaLink="false">http://tech-stuff.org/?p=128</guid>
		<description><![CDATA[A much dreaded MySQL error message during queries is &#8220;MySQL server has gone away&#8221;. An alternative message is &#8220;Lost connection to server during query&#8221;. This is a strange problem which afflicts a wide variety of PHP software including but not limited to WordPress. There are several causes for it. Let&#8217;s look at the common and [...]]]></description>
			<content:encoded><![CDATA[<p>A much dreaded MySQL error message during queries is &#8220;MySQL server has gone away&#8221;. An alternative message is &#8220;Lost connection to server during query&#8221;. This is a strange problem which afflicts a wide variety of <strong></strong><strong>PHP</strong> software including but not limited to WordPress. There are several causes for it. Let&#8217;s look at the common and some rare causes and what you can do to fix it.</p>
<p><span id="more-2041"> </span></p>
<p>The most common causes are:<br />
1. The server timed out and closed the connection. By default, the server closes the connection after 8 hours or 28800 seconds if nothing has happened. You can change the time limit by setting the wait_timeout variable when you start mysqld via your server&#8217;s /etc/my.cnf (on Linux; locate the file in installation directory on windows) as well. This mostly affects persistent connections; connections opened using mysql_pconnect() in <strong>PHP</strong>. It can also affect pooled connections from say any server side connection pooling.</p>
<p>2. Another common reason to receive the MySQL server has gone away error is because you have issued a &#8220;close&#8221; on your MySQL connection and then tried to run a query on the closed connection. This is a simple logic problem. Are you sharing the connection across multiple threads?</p>
<p>3. You got a timeout from the TCP/IP connection on the client side. This may happen if you have been using the commands: mysql_options(…, MYSQL_OPT_READ_TIMEOUT,…) or mysql_options(…, MYSQL_OPT_WRITE_TIMEOUT,…). In this case increasing the timeout, as described above, may help solve the problem.</p>
<p>4. You have encountered a timeout on the server side and the <a href="http://blog.taragana.com/index.php/archive/how-to-enable-disable-auto-reconnect-in-mysql/">automatic reconnection</a> in the client is disabled. Please refer to the article linked above for details and solution.</p>
<p>5. You can also get these errors if you send a query to the server that is incorrect or too large. If mysqld receives a packet that is too large or out of order, it assumes that something has gone wrong with the client and closes the connection. If you need big queries (for example, if you are working with big BLOB columns), you can increase the query limit by setting the server&#8217;s max_allowed_packet variable, which has a default value of 1MB. You may also need to increase the maximum packet size on the client end. More information on setting the packet size is given in Section B.1.2.9, “Packet too large”.</p>
<p>6. An INSERT or REPLACE statement that inserts a great many rows can also cause these sorts of errors. Either one of these statements sends a single request to the server irrespective of the number of rows to be inserted; thus, you can often avoid the error by reducing the number of rows sent per INSERT or REPLACE.</p>
<p>7. You also get a lost connection if you are sending a packet 16MB or larger if your client is older than 4.0.8 and your server is 4.0.8 and above, or the other way around.</p>
<p>Few rare causes are:<br />
1. Rarely the db administrator may have killed the running thread with a KILL statement or a mysqladmin kill command.</p>
<p>2. A client application running on a different host does not have the necessary privileges to connect to the MySQL server from that host.</p>
<p>3. You are using a Windows client and the server had dropped the connection (probably because wait_timeout expired) before the command was issued. The problem on Windows is that in some cases MySQL doesn&#8217;t get an error from the OS when writing to the TCP/IP connection to the server, but instead gets the error when trying to read the answer from the connection.</p>
<p>4. Prior to MySQL 5.0.19, even if the reconnect flag in the MYSQL structure is equal to 1, MySQL does not automatically reconnect and re-issue the query as it doesn&#8217;t know if the server did get the original query or not.</p>
<p>5. It is also possible to see this error if hostname lookups fail (for example, if the DNS server on which your server or network relies goes down). This is because MySQL is dependent on the host system for name resolution, but has no way of knowing whether it is working — from MySQL&#8217;s point of view the problem is indistinguishable from any other network timeout.</p>
<p>6. You may also see the MySQL server has gone away error if MySQL is started with the –skip-networking option.</p>
<p>7. You can also encounter this error with applications that fork child processes, all of which try to use the same connection to the MySQL server. This can be avoided by using a separate connection for each child process.</p>
<p>8. Another networking issue that can cause this error occurs if the MySQL port (default 3306) is blocked by your firewall, thus preventing any connections at all to the MySQL server.</p>
<p>9. You have encountered a bug where the server died while executing the query. <a href="http://dev.mysql.com/doc/refman/5.0/en/gone-away.html">Source</a></p>
<p>It takes time and effort to identify and solve these problems. Speaking from experience it is often not what it looks in the first place. Don&#8217;t assume anything, diligently rule out all possibilities till you solve it.</p>
]]></content:encoded>
			<wfw:commentRss>http://tech-stuff.org/mysql-tip-mysql-server-has-gone-away-or-lost-connection-to-server-during-query/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Minified using disk: basic
Page Caching using disk: enhanced
Database Caching 2/20 queries in 0.016 seconds using disk: basic
Object Caching 247/257 objects using disk: basic

Served from: tech-stuff.org @ 2012-02-05 22:21:49 -->
