{"id":2747,"date":"2023-03-20T14:54:44","date_gmt":"2023-03-20T14:54:44","guid":{"rendered":"http:\/\/kedar.nitty-witty.com\/?p=2747"},"modified":"2024-02-12T07:49:22","modified_gmt":"2024-02-12T07:49:22","slug":"mysql-8-timestamp-cannot-be-null-and-explicit_defaults_for_timestamp","status":"publish","type":"post","link":"https:\/\/kedar.nitty-witty.com\/blog\/mysql-8-timestamp-cannot-be-null-and-explicit_defaults_for_timestamp","title":{"rendered":"MySQL 8 &#8211; timestamp cannot be null and explicit_defaults_for_timestamp"},"content":{"rendered":"\n<p>A friend&#8217;s application started failing with MySQL causing error about timestamp columns and it needs urgent fixing from the database side. A timestamp column was not accepting the null values and they were sort of down.<\/p>\n\n\n\n<pre class=\"wp-block-code has-xl-font-size\"><code>ERROR 1048 (23000): Column 'start_date' cannot be null<\/code><\/pre>\n\n\n\n<p>In this blog we will learn how MySQL is telling you, to follow what you say. When you define the column NOT NULL it means NOT NULL and (latest) MySQL is going to error if you don&#8217;t respect that.<\/p>\n\n\n\n<p>Also, there&#8217;s golden bullet to solve any production problem like this towards the end of this blog: make sure to read through completely.<\/p>\n\n\n\n<p>He shared the table definition with me and it looked pretty OK to me.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE TABLE notnull (\nid int NOT NULL AUTO_INCREMENT,\nstart_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,\nend_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,\nPRIMARY KEY (id)\n) ENGINE=InnoDB;<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Testing timestamp behaviour in MySQL 5.7<\/h3>\n\n\n\n<p>I started my lab (Percona Server 5.7) and tested:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql&gt; CREATE TABLE notnull ( id int NOT NULL AUTO_INCREMENT, start_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, end_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ;\nQuery OK, 0 rows affected (0.01 sec)\n\nmysql&gt; insert into notnull values (null, null, null);\nQuery OK, 1 row affected (0.00 sec)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql&gt; select * from notnull;\n+----+---------------------+---------------------+\n| id | start_date | end_date |\n+----+---------------------+---------------------+\n| 1 | 2023-02-24 12:37:35 | 2023-02-24 12:37:35 |\n+----+---------------------+---------------------+\n1 row in set (0.00 sec)<\/code><\/pre>\n\n\n\n<!--more-->\n\n\n\n<p>It works! What&#8217;s the problem? I further asked his SQL_MODE and MySQL Version. <\/p>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">A workaround that wasn&#8217;t good<\/h4>\n\n\n\n<p>We also discussed further about this issue and one solution proposed was using a BEFORE TRIGGER. Since the timestamp value are inserted as NULL, we will provide a NON NULL value before the INSERT is actually processed.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE TRIGGER start_date_trigger BEFORE INSERT ON notnull FOR EACH ROW SET NEW.start_date = CURRENT_TIMESTAMP;\nCREATE TRIGGER end_date_trigger BEFORE INSERT ON notnull FOR EACH ROW SET NEW.end_date = CURRENT_TIMESTAMP;\n\nMySQL &#91;test]&gt; insert into notnull values(null,null,null);\nQuery OK, 1 row affected (0.051 sec)\n\nMySQL &#91;test]&gt; select * from notnull ;\n| 29 | 2023-02-24 11:57:34 | 2023-02-24 11:57:35 |\n| 30 | 2023-02-24 13:33:24 | 2023-02-24 13:33:24 |<\/code><\/pre>\n\n\n\n<p>But we both agreed, that it&#8217;s not a correct way.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reproducing the error on MySQL 8<\/h3>\n\n\n\n<p>Anyways, he further confirmed that the Server version was 8.0.26 and SQL_MODE remained NO_ENGINE_SUBSTITUTION. So, I started MySQL 8 lab, and retried to produce the error and succeeded to generate a failure.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql &#91;localhost:8028] {msandbox} (test) &gt; insert into notnull values (null,null,null);\n<strong>ERROR 1048 (23000): Column 'start_date' cannot be null<\/strong><\/code><\/pre>\n\n\n\n<p>On MySQL 8 we see what&#8217;s the problem. One of the many changes that has been implemented in MySQL 8 (8.0.22+) is the alteration of the default value of explicit_defaults_for_timestamp from OFF to ON. This may seem like a small change, but it has significant implications for the way NOT NULL and default timestamps work.<br>The note says: As of MySQL 8.0.22, attempting to insert NULL into a generated column declared as TIMESTAMP NOT NULL is rejected with an error.<\/p>\n\n\n\n<p>So the (temporary) solution to the problem here is to make the change and retain in config.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>SET GLOBAL explicit_defaults_for_timestamp = OFF;<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Explanation of the issue<\/h3>\n\n\n\n<p>Let&#8217;s understand a little more about explicit_defaults_for_timestamp. This system variable controls whether or not MySQL should use explicit DEFAULT expressions for timestamp columns that have no explicit DEFAULT value defined.<\/p>\n\n\n\n<p>When explicit_defaults_for_timestamp is set to OFF, MySQL does not use explicit DEFAULT expressions for timestamp columns, and instead, it assigns a value of &#8216;0000-00-00 00:00:00&#8217; if the column is defined as NOT NULL, or NULL if it&#8217;s defined as NULLABLE.<\/p>\n\n\n\n<p>However, with the variable set to ON, MySQL uses explicit DEFAULT expressions for timestamp columns, even if they are not explicitly defined.<\/p>\n\n\n\n<p>As of MySQL 8.0.22, attempting to insert NULL into a generated column declared as TIMESTAMP NOT NULL is rejected with an error.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql &#91;localhost:8028] {msandbox} (test) &gt; set explicit_defaults_for_timestamp=ON;\nQuery OK, 0 rows affected (0.00 sec)\n\nmysql &#91;localhost:8028] {msandbox} (test) &gt; insert into notnull values (2,null,null);\nERROR 1048 (23000): Column 'start_date' cannot be null<\/code><\/pre>\n\n\n\n<p>But when we turn this variable OFF:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql &#91;localhost:8028] {msandbox} (test) &gt; set explicit_defaults_for_timestamp=OFF;\nQuery OK, 0 rows affected, 1 warning (0.00 sec)\n\nmysql &#91;localhost:8028] {msandbox} (test) &gt; show warnings;\n+---------+------+------------------------------------------------------------------------------------------+\n| Level | Code | Message |\n+---------+------+------------------------------------------------------------------------------------------+\n<strong>| Warning | 1287 | 'explicit_defaults_for_timestamp' is deprecated and will be removed in a future release. |<\/strong>\n+---------+------+------------------------------------------------------------------------------------------+\n\nmysql &#91;localhost:8028] {msandbox} (test) &gt; insert into notnull values (3,null,null);\nQuery OK, 1 row affected (0.00 sec)\n\nmysql &#91;localhost:8028] {msandbox} (test) &gt; select * from notnull;\n+----+---------------------+---------------------+\n| id | start_date | end_date |\n+----+---------------------+---------------------+\n| 1 | 2023-03-19 19:16:01 | 2023-03-19 19:16:01 |\n| 3 | 2023-03-19 19:18:12 | 2023-03-19 19:18:12 |\n+----+---------------------+---------------------+<\/code><\/pre>\n\n\n\n<p>I hope you did not miss the WARNING above <em>&#8220;explicit_defaults_for_timestamp&#8217; is deprecated and will be removed in a future release.&#8221;<\/em>. <\/p>\n\n\n\n<p>Also, not providing the column names should allow it to pick up the defaults.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mysql &#91;localhost:8028] {msandbox} (test) &gt; insert into notnull(id) values (null);\nQuery OK, 1 row affected (0.00 sec)\n\nmysql &#91;localhost:8028] {msandbox} (test) &gt; select * from notnull;\n+----+---------------------+---------------------+\n| id | start_date | end_date |\n+----+---------------------+---------------------+\n| 1 | 2023-03-19 19:16:01 | 2023-03-19 19:16:01 |\n| 3 | 2023-03-19 19:18:12 | 2023-03-19 19:18:12 |\n| 4 | 2023-03-19 19:20:20 | 2023-03-19 19:20:20 |\n+----+---------------------+---------------------+\n3 rows in set (0.00 sec)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Conclusion<\/h3>\n\n\n\n<p>Well, the issue was solved for my friend by setting explicit_defaults_for_timestamp OFF but remember that this MySQL variable is going away. So respect your definition &#8211; NOT NULL means NOT NULL.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><img decoding=\"async\" width=\"300\" height=\"217\" src=\"http:\/\/kedar.nitty-witty.com\/wp-content\/uploads\/2023\/03\/image-300x217.png\" alt=\"\" class=\"wp-image-2748\" srcset=\"https:\/\/kedar.nitty-witty.com\/blog\/wp-content\/uploads\/2023\/03\/image-300x217.png 300w, https:\/\/kedar.nitty-witty.com\/blog\/wp-content\/uploads\/2023\/03\/image-768x556.png 768w, https:\/\/kedar.nitty-witty.com\/blog\/wp-content\/uploads\/2023\/03\/image.png 948w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/figure>\n<\/div>\n\n\n<p>Refer the <a href=\"https:\/\/dev.mysql.com\/worklog\/task\/?id=9687\" target=\"_blank\" rel=\"noopener nofollow\" title=\"\">MySQL worklog<\/a> for detail information and high level architecture for the fixture. <\/p>\n\n\n\n<p>Okay so for those who believed in golden bullet, :faceplam: , but I surely have a secret pro-tip.<\/p>\n\n\n\n<p>How did this issue start? My friend confirmed that they did a MySQL Migration. They migrated from MariaDB to MySQL  8, on a Friday! Without testing! If you can&#8217;t believe that a day is Friday, use any of the methods used here to <a href=\"https:\/\/kedar.nitty-witty.com\/blog\/providing-friday-proofs-to-mysql-consultants\" target=\"_blank\" rel=\"noopener\" title=\"\">prove a Friday<\/a> and act accordingly. <\/p>\n\n\n\n<p>ProTip: <strong>Never<\/strong>, I repeat, Never  do any production change on a Friday, your engineers will love you. <\/p>\n","protected":false},"excerpt":{"rendered":"A friend&#8217;s application started failing with MySQL causing error about timestamp columns and it needs urgent fixing from the database side. A timestamp column was not accepting the null values&hellip;\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[8,867,377],"tags":[527,528,525,427,691,505,526,529],"class_list":{"0":"post-2747","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-mysql","7":"category-mysql-upgrade","8":"category-mysql-articles","9":"tag-column-cannot-be-null","10":"tag-error-1024-2300","11":"tag-explicit_defaults_for_timestamp","12":"tag-mysql","13":"tag-mysql-8-upgrade-issue","14":"tag-mysql8","15":"tag-timestamp","16":"tag-timestamp-not-null"},"aioseo_notices":[],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/posts\/2747","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/comments?post=2747"}],"version-history":[{"count":13,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/posts\/2747\/revisions"}],"predecessor-version":[{"id":3279,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/posts\/2747\/revisions\/3279"}],"wp:attachment":[{"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/media?parent=2747"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/categories?post=2747"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kedar.nitty-witty.com\/blog\/wp-json\/wp\/v2\/tags?post=2747"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}