Search code examples
storageceph

ceph mount failed with (95) Operation not supported

I installed ceph on servers "A" and "B" and I would like to mount it from "C" or "D" servers.

But I faced below error.

ceph-fuse[4628]: ceph mount failed with (95) Operation not supported

My server configuration is as follow.

A Server: ubunt16.04(ceph-server) 10.1.1.54
B Server: ubuntu16.04(ceph-server) 10.1.1.138
C Server: AmazonLinux(clinet)
D Server: ubuntu16.04(client)

and ceph.conf

[global]
fsid = 44f299ac-ff11-41c8-ab96-225d62cb3226
mon_initial_members = node01, node02
mon_host = 10.1.1.54,10.1.1.138
auth cluster required = none
auth service required = none
auth client required = none
auth supported = none
osd pool default size = 2
public network = 10.1.1.0/24

Ceph is also installed correctly.

ceph health

HEALTH_OK

ceph -s 

  cluster 44f299ac-ff11-41c8-ab96-225d62cb3226
     health HEALTH_OK
     monmap e1: 2 mons at {node01=10.1.1.54:6789/0,node02=10.1.1.138:6789/0}
            election epoch 12, quorum 0,1 node01,node02
     osdmap e41: 2 osds: 2 up, 2 in
            flags sortbitwise,require_jewel_osds
      pgmap v100: 64 pgs, 1 pools, 306 bytes data, 4 objects
            69692 kB used, 30629 MB / 30697 MB avail
                  64 active+clean

An error occurred when using the ceph-fuse command.

sudo ceph-fuse -m 10.1.1.138:6789 /mnt/mycephfs/ --debug-auth=10 --debug-ms=10
ceph-fuse[4628]: starting ceph client
2017-11-02 08:57:22.905630 7f8cfdd60f00 -1 init, newargv = 0x55779de6af60 newargc=11
ceph-fuse[4628]: ceph mount failed with (95) Operation not supported
ceph-fuse[4626]: mount failed: (95) Operation not supported

I got an error saying "ceph mount failed with (95) Operation not supported"

I added an option "--auth-client-required=none"

sudo ceph-fuse -m 10.1.1.138:6789 /mnt/mycephfs/ --debug-auth=10 --debug-ms=10 --auth-client-required=none
ceph-fuse[4649]: starting ceph client
2017-11-02 09:03:47.501363 7f1239858f00 -1 init, newargv = 0x5597621eaf60 newargc=11

Behavior changed, There is no response here.

I got an below error if ceph-fuse command is not used.

sudo mount -t ceph 10.1.1.138:6789:/ /mnt/mycephfs

can't read superblock

Somehow, it seems necessary to authenticate with client even with "auth supported = none"

In that case, how could you pass authentication form servers "c" or "d"?

Please let me know, If there is possible cause other than authentication.

Solution

  • I solved this problem by fixing three settings.

    1.

    The auth settings in ceph.conf returned as follows

    auth cluster required = cephx
auth service required = cephx
auth client required = cephx

    2.

    public network was wrong.

    public network = 10.1.1.0/24

    ↓

    public network = 10.0.0.0/8

    my client ip address was 10.1.0.238... It was a stupid mistake.

    3.

    I changed secret option to secretfile option and everything was fine.

    In this case, it was failed.

    sudo mount -t ceph 10.1.1.138:6789:/ /mnt/mycephfs -o name=client.admin,secret=`sudo ceph-authtool -p /etc/ceph/ceph.client.admin.keyring`

    output:

    mount error 1 = Operation not permitted

    but in this case, It was success.

    sudo mount -vvvv -t ceph 10.1.1.138:6789:/ /mnt/mycephfs -o name=admin,secretfile=admin.secret

    output:

    parsing options: rw,name=admin,secretfile=admin.secret
mount: error writing /etc/mtab: Invalid argument

    ※ Invalid argument The error seems to be ignored.

    Apparently, Both are the same key.

    sudo ceph-authtool -p /etc/ceph/ceph.client.admin.keyring
AQBd9f9ZSL46MBAAqwepJDC5tuIL/uYp0MXjCA==


cat admin.secret 
AQBd9f9ZSL46MBAAqwepJDC5tuIL/uYp0MXjCA==

    I don't know reason,but I could mount using secretfile option.