Speeding up "Mapcache_seed" with Mapserver

I'm using mapcache_seed in the mapcache package to create a large image cache from calling my mapserver WMS, with vectors.

Currently, this is the command I'm using:

sudo -u www-data mapcache_seed -c mapcache.xml -g WGS84 -n 8 -t Test -e\ [Foo,Bar,Baz,Fwee] -M 8,8 -z 12,13 --thread-delay 0 --rate-limit 10000

Where www-data is my Nginx system-user, mapcache.xml is my config, WGS84 is my SRS, -n 8 is my logical thread count (on a i7-6700HQ at 3200 MHz), -z 12,13 is one zoom level that needs to be seeded, thread-delay is off, and tile rate creation is set to 10000.

However, I only (max) get 50% total CPU utilization and most times only a single core goes above 50%. And an average of 500 tiles per second -- independent of how many threads or processes I specify. I've been trying to get all zoom-levels (4 to 27) seeded for the last couple of days, but I've only managed to get through 4-12, before being severely bottle-necked at a mere 3GB of a couple million tiles.

Memory utilization is at a stable 2.4% for 8GB PC4-2133 for mapcache_seed (0.5 for the WMS). Write speeds are at 100 MB/s, no-buffer write is also 100 MB/s, while buffered+cache is at 6.7-8.7 GB/s on a SATA III 1TB HDD. I have another SSD drive on my machine that gets 6 GB/s write and 8 GB/s read, but it's too small for storage and I'm afraid of drive failure from too many writes.

The cached tiles are around 4KB each and that means I get around 2MB worth of tiles every second. The majority of them aren't even tiles, but symlinks to a catch-all blank tile for empty tiles.

How would I go about speeding this process up? Messing with threads and limits, through mapcache_seed, does not make any discernible difference. This is also on a Debian Wheezy machine.

This is also being run through fast-cgi, using 256x256 px images, and disk cache with a restricted extent to a single country (otherwise mapcache starts generating nothing but symlinks to blank tiles, because more than 90% of the world is blank!)

Mapserver mapfile (redacted):

MAP
   NAME         "MAP"
   SIZE         1200 800
   EXTENT       Foo Bar Baz Fwee
   UNITS        DD
   SHAPEPATH        "."
   IMAGECOLOR   255 255 255
   IMAGETYPE        PNG
   WEB
     IMAGEPATH  "/path/to/image"
     IMAGEURL   "/path/to/imageurl"
     METADATA
        "wms_title"                     "MAP"
        "wms_onlineresource"            "http://localhost/cgi-bin/mapserv?MAP=/path/to/map.map"
        "wms_srs"                       "EPSG:4326"
        "wms_feature_info_mime_type"    "text/plain"
        "wms_abstract"                  "Lorem ipsum"
        "ows_enable_request"            "*"
        "wms_enable_request"            "*"
     END
   END

PROJECTION
  "init=epsg:4326"
END

   LAYER
     NAME       base
     TYPE       POLYGON
     STATUS     OFF
     DATA       polygon.shp
     CLASS
        NAME        "Polygon"
        STYLE
          COLOR     0 0 0
          OUTLINECOLOR  255 255 255
        END
     END
   END

   LAYER
     NAME       outline
     TYPE       LINE
     STATUS     OFF
     DATA       line.shp
     CLASS
       NAME     "Line"
       STYLE
         OUTLINECOLOR   255 255 255
       END
     END
   END  
END

mapcache.xml (redacted):

<?xml version="1.0" encoding="UTF-8"?>

<mapcache>
<source name="ms4wserver" type="wms">
<getmap>
  <params>
    <LAYERS>base</LAYERS>
        <MAP>/path/to/map.map</MAP>
  </params>
</getmap>
<http>
  <url>http://localhost/wms/</url>
</http>
  </source>

<cache name="disk" type="disk">
<base>/path/to/cache/</base>
<symlink_blank/>
  </cache>

  <tileset name="test">
<source>ms4wserver</source>
<cache>disk</cache>
<format>PNG</format>
<grid>WGS84</grid>
<metatile>5 5</metatile>
<metabuffer>10</metabuffer>
<expires>3600</expires>
  </tileset>

   <default_format>JPEG</default_format>

   <service type="wms" enabled="true">
 <full_wms>assemble</full_wms>
  <resample_mode>bilinear</resample_mode>
  <format>JPEG</format>
  <maxsize>4096</maxsize>
   </service>
   <service type="wmts" enabled="false"/>
   <service type="tms" enabled="false"/>
   <service type="kml" enabled="false"/>
   <service type="gmaps" enabled="false"/>
   <service type="ve" enabled="false"/>
   <service type="mapguide" enabled="false"/>
   <service type="demo" enabled="false"/>

   <errors>report</errors>
   <locker type="disk">
 <directory>/path/</directory>
 <timeout>300</timeout>
</locker>

</mapcache>

Solution

So for anyone coming across this ten years later, when what's left of the little documentation for these tools has rotted away, I messed with my mapcache and mapfile settings to get something better. It wasn't that my generation was too slow, it was that I was generating TOO MANY GODDAMN SYMLINKS to blank files. First, the mapfile extent was incorrect. Second, I was using the "WGS84" grid, which by default seeds ALL extents. That means 90% of all my tiles were just symlinks to a blank.png, and it ate up ALL of my inodes. I recommend using mkdir blank; rsync -a --delete blank/ /path/to/cache for a quick clearing of all that mess.

I fixed the above by taking the WGS84 specifications and changing the extent to the one I specified in my mapfile. Now, only my mapfile gets seeded. Lastly, I appended the grid XML element like so:

<grid restricted_extent="MAP FILE EXTENT HERE">GRIDNAME</grid>

With restricted_extent now it's certain that only my map gets seeded. I had over 100 million tiles, but they were all goddamn symlinks! Otherwise, I got a "Ran out of space" or something of some such. Even if df shows that the partition isn't full, it's misleading. Symlinks take up inode space, not logical space! To see inode space, run df -hi. I was at 100% inode space, but only 1% logical space on a 1TB drive -- filled with goddamn symlinks!