I'm using symfony 3, and have the following .htaccess file (attached to the very bottom of this message noted as 'problem .htaccess'). I'm using the website gtmetrix.com (which uses pagespeed) to optimize my site, but I'm getting the following error for my .css and .js files.
Specify a Vary: Accept-Encoding header
F (33)
SERVER HIGH
What's this mean?
The following publicly cacheable, compressible resources should have a "Vary: Accept-Encoding" header:
http://{{my website here}}/css/main.css
http://{{my website here}}/js/main.js
Before you mention that it says Header append Vary User-Agent which clearly does not say accept-encoding..
1.) (I have already looked at How to Specify "Vary: Accept-Encoding" header in .htaccess), which talks about how to set this.
2.) The code I have listed works on another site on my same server and I get no vary encoding error message., which means that the code I have should have worked anyways (and the other site has a main.css and main.js also). The only difference in that htaccess file is that it was for SLIM framework instead of symfony so it looked like the following below:
SLIM FRAMEWORK (OTHER SITE .HTACCESS)
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^ index.php [QSA,L]
AddType application/x-font-ttf .ttf
Header unset Pragma
FileETag None
Header unset ETag
<IfModule mod_deflate.c>
# Compress HTML, CSS, JavaScript, Text, XML and fonts
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
AddOutputFilterByType DEFLATE application/x-font
AddOutputFilterByType DEFLATE application/x-font-opentype
AddOutputFilterByType DEFLATE application/x-font-otf
AddOutputFilterByType DEFLATE application/x-font-truetype
AddOutputFilterByType DEFLATE application/x-font-ttf
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE font/opentype
AddOutputFilterByType DEFLATE font/otf
AddOutputFilterByType DEFLATE font/ttf
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml
# Remove browser bugs (only needed for really old browsers)
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
Header append Vary User-Agent
</IfModule>
## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access plus 1 year"
ExpiresByType font/ttf "access plus 1 year"
ExpiresByType application/x-font-ttf "access plus 1 year"
ExpiresByType image/jpeg "access plus 1 year"
ExpiresByType image/gif "access plus 1 year"
ExpiresByType image/png "access plus 1 year"
ExpiresByType text/css "access plus 1 month"
ExpiresByType application/pdf "access plus 1 month"
ExpiresByType text/x-javascript "access plus 1 month"
ExpiresByType application/x-shockwave-flash "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 year"
ExpiresDefault "access plus 2 days"
</IfModule>
## EXPIRES CACHING ##
Symfony 3 FRAMEWORK (The Problem .HTACCESS)
# Use the front controller as index file. It serves as a fallback solution when
# every other rewrite/redirect fails (e.g. in an aliased environment without
# mod_rewrite). Additionally, this reduces the matching process for the
# start page (path "/") because otherwise Apache will apply the rewriting rules
# to each configured DirectoryIndex file (e.g. index.php, index.html, index.pl).
DirectoryIndex app.php
# By default, Apache does not evaluate symbolic links if you did not enable this
# feature in your server configuration. Uncomment the following line if you
# install assets as symlinks or if you experience problems related to symlinks
# when compiling LESS/Sass/CoffeScript assets.
# Options FollowSymlinks
# Disabling MultiViews prevents unwanted negotiation, e.g. "/app" should not resolve
# to the front controller "/app.php" but be rewritten to "/app.php/app".
<IfModule mod_negotiation.c>
Options -MultiViews
</IfModule>
## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access plus 1 year"
ExpiresByType image/jpeg "access plus 1 year"
ExpiresByType image/gif "access plus 1 year"
ExpiresByType image/png "access plus 1 year"
ExpiresByType text/css "access plus 1 month"
ExpiresByType application/pdf "access plus 1 month"
ExpiresByType text/x-javascript "access plus 1 month"
ExpiresByType text/javascript "access plus 1 month"
ExpiresByType text/js "access 1 week"
ExpiresByType application/javascript "access 1 week"
ExpiresByType application/x-shockwave-flash "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 year"
ExpiresDefault "access plus 2 days"
</IfModule>
## EXPIRES CACHING ##
<IfModule mod_rewrite.c>
RewriteEngine On
# Determine the RewriteBase automatically and set it as environment variable.
# If you are using Apache aliases to do mass virtual hosting or installed the
# project in a subdirectory, the base path will be prepended to allow proper
# resolution of the app.php file and to redirect to the correct URI. It will
# work in environments without path prefix as well, providing a safe, one-size
# fits all solution. But as you do not need it in this case, you can comment
# the following 2 lines to eliminate the overhead.
RewriteCond %{REQUEST_URI}::$1 ^(/.+)/(.*)::\2$
RewriteRule ^(.*) - [E=BASE:%1]
# Sets the HTTP_AUTHORIZATION header removed by Apache
RewriteCond %{HTTP:Authorization} .
RewriteRule ^ - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
# Redirect to URI without front controller to prevent duplicate content
# (with and without `/app.php`). Only do this redirect on the initial
# rewrite by Apache and not on subsequent cycles. Otherwise we would get an
# endless redirect loop (request -> rewrite to front controller ->
# redirect -> request -> ...).
# So in case you get a "too many redirects" error or you always get redirected
# to the start page because your Apache does not expose the REDIRECT_STATUS
# environment variable, you have 2 choices:
# - disable this feature by commenting the following 2 lines or
# - use Apache >= 2.3.9 and replace all L flags by END flags and remove the
# following RewriteCond (best solution)
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^app\.php(?:/(.*)|$) %{ENV:BASE}/$1 [R=301,L]
# If the requested filename exists, simply serve it.
# We only want to let Apache serve files and not directories.
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^ - [L]
# Rewrite all other queries to the front controller.
RewriteRule ^ %{ENV:BASE}/app.php [L]
</IfModule>
<IfModule !mod_rewrite.c>
<IfModule mod_alias.c>
# When mod_rewrite is not available, we instruct a temporary redirect of
# the start page to the front controller explicitly so that the website
# and the generated links can still be used.
RedirectMatch 302 ^/$ /app.php/
# RedirectTemp cannot be used instead
</IfModule>
</IfModule>
# These rules need to be at the bottom!
# Make it aware of True Type Fonts for Web Fonts.
AddType application/x-font-ttf .ttf
Header unset Pragma
FileETag None
Header unset ETag
<IfModule mod_deflate.c>
#The following line is enough for .js and .css
AddOutputFilter DEFLATE js css
# Compress HTML, CSS, JavaScript, Text, XML and fonts
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
AddOutputFilterByType DEFLATE application/x-font
AddOutputFilterByType DEFLATE application/x-font-opentype
AddOutputFilterByType DEFLATE application/x-font-otf
AddOutputFilterByType DEFLATE application/x-font-truetype
AddOutputFilterByType DEFLATE application/x-font-ttf
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE font/opentype
AddOutputFilterByType DEFLATE font/otf
AddOutputFilterByType DEFLATE font/ttf
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/javascript
AddOutputFilterByType DEFLATE text/js
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml
# Remove browser bugs (only needed for really old browsers)
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
Header append Vary User-Agent
</IfModule>
Question is: What is different about symfony framework or their .htaccess file that will NOT allow me to use the Header append Vary User-Agent command?
UPDATE
So after doing some experimentation I modified symfony's .htaccess file to the same as slims, the only difference I had to do was add the line DirectoryIndex app.php to the slims .htaccess file and then, tell it to rewrite to app.php instead of index.php, but still the error persists. This tells me its probably not the htaccess files but maybe the way I'm going about creating my assets?
In code it looks like this in a TWIG template.
{% spaceless %}
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<title>{% block title %}Welcome!{% endblock %}</title>
{% block stylesheets %}
<link href="{{ asset('css/main.css') }}" rel="stylesheet" />
{% endblock %}
<link rel="icon" type="image/x-icon" href="{{ asset('favicon.ico') }}" />
</head>
<body>
{% block body %}{% endblock %}
{% block javascripts %}
<script src="{{ asset('js/main.js') }}"></script>
{% endblock %}
</body>
</html>
{% endspaceless %}
Maybe because I'm using asset its doing something to the encoders for that resource somehow in the engine? I'm not sure how it would do that, but not sure where else to look for the moment.
Update #2
Ok made big headway in this mystery, I got the vary-encoding message to go away but I'm not entirely sure why, but I do know the file size of the main.css file is important.
When the main.css has the following it will fail to specify a vary:accept encoding header? Maybe because it was too small to compress in the first place?
.someclass{color:red;}.heythere{color:blue;}.heythereagain{color:blue;width:200px}.heytheretwice{color:green;height:50px;}
However give it the following main.css contents instead and the message goes away on gtmetrix.
a,abbr,acronym,address,applet,article,aside,audio,b,big,blockquote,body,canvas,caption,center,cite,code,dd,del,details,dfn,div,dl,dt,em,embed,fieldset,figcaption,figure,footer,form,h1,h2,h3,h4,h5,h6,header,hgroup,html,i,iframe,img,ins,kbd,label,legend,li,mark,menu,nav,object,ol,output,p,pre,q,ruby,s,samp,section,small,span,strike,strong,sub,summary,sup,table,tbody,td,tfoot,th,thead,time,tr,tt,u,ul,var,video{margin:0;padding:0;border:0;font-size:100%;font:inherit;vertical-align:baseline}article,aside,details,figcaption,figure,footer,header,hgroup,menu,nav,section{display:block}body{line-height:1}ol,ul{list-style:none}blockquote,q{quotes:none}blockquote:after,blockquote:before,q:after,q:before{content:'';content:none}table{border-collapse:collapse;border-spacing:0}.red{background-color:rgba(255,0,0,.75)}.blue{background-color:rgba(0,0,255,.75)}.green{background-color:rgba(0,255,0,.75)}.purple{background-color:rgba(255,0,255,.75)}.orange{background-color:rgba(255,150,0,.75)}.yellow{background-color:rgba(255,255,0,.75)}.black{background-color:rgba(0,0,0,.75)}.hide{display:none}.inline_block_fix:before{display:inline-block;content:'';height:100%;width:0;vertical-align:middle}.vertical_align{vertical-align:middle;display:inline-block;height:100%;width:0}.header{height:70px;color:#FFF;background-color:#2c3e50;font-size:16px;text-align:right;font-family:ProximaNova;text-align:center}.header a{color:#fff;text-decoration:none}.header .fa-phone{color:#42c044;padding-left:10px;padding-right:10px;font-size:24px!important;vertical-align:middle}.mobile_menu{display:inline-block;vertical-align:middle;font-size:24px!important;padding-right:10px;cursor:pointer}.phonebox{display:inline-block;vertical-align:middle;margin-right:0;border-radius:10px 10px 10px 10px;-moz-border-radius:10px 10px 10px 10px;-webkit-border-radius:10px 10px 10px 10px;border:0 solid #000;background-color:#42c044;height:41px;width:210px;font-size:26px;font-weight:700;text-align:center}.phonebox .phonetext{display:inline-block;vertical-align:middle}.callustext{display:none;vertical-align:middle}@media all and (min-width:600px){.callustext{display:inline-block}.mobile_menu{display:none!important}.header{text-align:right}.phonebox{margin-right:40px}}.footer{background-color:#2c3e50;text-align:center;font-family:ProximaNova;color:#fff;padding-top:30px}.footer .copyright_area{height:60px;line-height:60px;background-color:#33383c;color:#aaa;font-size:12px;margin-top:30px}.footer .copyright_area .copyright_links{padding-left:20px}.footer .copyright_area .copyright_links a{color:#aaa;padding-left:2px;padding-right:2px}.footer .whitebox{display:inline-block;vertical-align:middle;margin-left:20px;margin-right:20px;padding-top:20px;padding-bottom:15px;padding-left:30px;padding-right:30px;background-color:#fff;border-radius:5px;text-align:center}.footer .whitebox .footer_logo{display:inline-block;background-image:url(/images/logo-hires-opt.png);background-repeat:no-repeat;background-position:top;background-attachment:scroll;background-size:196px 75px;height:95px;width:196px}.footer .whitebox .bar{border-bottom:2px solid #485867;margin-bottom:20px}.footer .whitebox .asp{display:inline-block;background-image:url(/images/asp-opt.jpg);background-size:95px 75px;background-repeat:no-repeat;height:75px;width:95px}.footer .whitebox .cpo{display:inline-block;background-image:url(/images/cpo-opt.jpg);background-size:75px 75px;background-repeat:no-repeat;background-position:center;height:75px;width:100px}.footer .links_area{display:inline-block;vertical-align:middle;text-align:left;font-size:18px;margin-left:20px;margin-right:20px;width:285px;margin-top:30px}.footer .links_area .links_area_title{font-size:28px;font-weight:700;margin-bottom:20px}.footer .links_area ul li{margin-left:10px;margin-top:10px}.footer .links_area ul li i{width:10px;margin-right:20px}.footer .links_area a{color:#fff;text-decoration:none;border-bottom:2px solid #777F88}@font-face{font-family:ProximaNova;src:url(/fonts/proximanova-regular-webfont.ttf)}@font-face{font-family:ProximaNovaAltBold;src:url(/fonts/proxima_nova_alt_bold-webfont.ttf)}
I still have the error on the javascript but I'm going to keep adding random functions to my javascript file to see if that makes the error go away. Currently my JS file is:
function test(){alert('test');}function test2(){alert('test2');function test3(){alert('test3');alert('another test');}}
Update #3:
Indeed the file size was the culprit, changing my js file to..
function test(){alert('test');}function test2(){alert('test2');}function test3(){alert('test3');alert('another test');}function test4(){alert('test4');alert('another test4');}function test5(){alert('test5');alert('another test5');}function test6(){alert('test6');alert('another test6');}function test7(){alert('test7');alert('another test7');}function test8(){alert('test8');alert('another test8');}function test9(){alert('test9');alert('another test9');}function test10(){alert('test10');alert('another test10');}function test11(){alert('test11');alert('another test11');}function test12(){alert('test12');alert('another test12');}function test13(){alert('test13');alert('another test13');}function test14(){alert('test14');alert('another test14');}function test15(){alert('test15');alert('another test15');}function test16(){alert('test16');alert('another test16');}function test17(){alert('test17');alert('another test17');}function test18(){alert('test18');alert('another test18');}function test19(){alert('test19');alert('another test19');}function test20(){alert('test20');alert('another test20');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');alert('hello');}
was enough to twigger the header vary encoding error to go away.
Update #4
Was reading this page (https://www.fastly.com/blog/best-practices-for-using-the-vary-header), and apparently if you are using mod_deflate the correct Vary header is automatically added to your responses. So its very likely that mod_deflate was overwriting my suggested change when the main.css and main.js files were small.
So new question:
Why does the file size matter when trying to add the Header append Vary User-Agent command? Should I just ignore the speed optimization suggestion from gtmetrix (pagespeed) in this case? Or if mod_deflate is overwriting, is this even a problem? If so how do I fix the overwriting of the header that mod_deflate causes?
So after much research and deliberation I have come to the conclusion that mod_deflate is the culprit here.
According to apache it's self: http://httpd.apache.org/docs/2.2/mod/mod_deflate.html
The mod_deflate module sends a Vary: Accept-Encoding HTTP response header to alert proxies that a cached response should be sent only to clients that send the appropriate Accept-Encoding request header. This prevents compressed content from being sent to a client that will not understand it.
The key here is understanding how compression works. When you look at an HTTP Request from a users browser for example you might see this. (Use google chromes inspector or Firefox's firebug to see these).
GET /css/main.css HTTP/1.1
Host: dev.yoursitehere.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: text/css,*/*;q=0.1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
Referer: http://dev.yoursitehere.com
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,nb;q=0.6
Cookie: PHPSESSID=2fd616076481f661975f7de2a92a0016
Notice this line: Accept-Encoding: gzip, deflate, sdch
This is saying to the server, I can uncompress gzip, deflate and sdch compression formats if you send them back to me.
The server does NOT have to send back compressed results. In fact this is what was happening in my situation when the file size was to small.
8096 bytes is the default set by the apache deflate moduels DeflateBufferSize
This means if the content you want to server is greater than 8096 bytes then it will compress the results, and according to my .htaccess file, I was using the DEFLATE method of compression.
Only once the file size was big enough did I notice the response headers looking like this.
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 29 Dec 2015 20:25:28 GMT
Content-Type: text/css
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Last-Modified: Tue, 29 Dec 2015 19:31:30 GMT
Cache-Control: max-age=2592000
Expires: Thu, 28 Jan 2016 20:25:28 GMT
Content-Encoding: gzip
Normally the Vary:Accept-Encoding response header was not there when the file size of the main.css or main.js file was to small, but once it was past the amount that was worthy of compressing the vary header was automatically added by the mod_deflate module.
That header tells chrome that this is NOT a normal uncompressed file but a file that needs to be uncompressed. In this particular case, content-encoding was set to gzip. This means it compressed the results before sending back to me.
https://en.wikipedia.org/wiki/Gzip
According to this, gzip uses the DEFLATE algorithm, (which my browser said it could handle in my request header with the line Accept-Encoding: gzip, deflate, sdch
Thus the browser was able to reconstruct it correctly.
This is why file size matters before the vary header is appended, and what the vary header actually means in this context.