Codeigniter still shows index.php after forcing https

Question

Just launched my site with an SSL certificate. When I go to the https site it works perfectly. However if I force just http, even after configuring htaccess, it forwards me to the https site, but it adds index.php?/ to the end of the url. For example if I go to http://www.my-site.com it redirects to https://www.my-site.com/index.php?/. Of course the site works fine, its just a bit of an eyesore in the url field.

base_url variable is as shown:

$config['base_url'] = 'http' . ((isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] == 'on') ? 's' : '').'://'.$_SERVER['HTTP_HOST'].str_replace('//','/',dirname($_SERVER['PHP_SELF']).'/');

My .htaccess looks like this:

RewriteEngine on
RewriteCond $1 !^(index\.php?|_assets|robots\.txt|sitemap\.xml|favicon\.ico?)
RewriteRule ^(.*)$ /index.php?/$1 [L]
RewriteCond %{HTTPS} off
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

I added the last 2 lines after researching how to force https when a visitor visits the site with http.

What am I missing? htaccess is not my strong suit. It's shared access otherwise I'd edit my httpd.conf

EDIT: Updated htaccess looks like this:

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?/$1
RewriteCond %{HTTPS} off
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

This technically works, but for some reason when I access the site using http it redirects to the https version but concatenates another copy of my URI to the end. For example if I go to http://example.com/products it redirects to https://example.com/products?/products.

Answer 1

The third line of htaccess should be:
RewriteRule ^(.*)$ index.php/$1
1. The question mark (?) shouldn't be there, anyway.
2. The [L] flag means last. When this rule is applied, no other directive runs.
So, the fifth line never runs, but this fifth line is the one which removes the index.php from the url.