PHP ldap_search size limit exceeded

Question

I'm quite new to querying Microsoft's Active Directory and encountering some difficulties:

The AD has a size limit of 1000 elements per request. I cannot change the size limit. PHP does not seem to support paging (I'm using version 5.2 and there's no way of updating the production server.)

I've so far encountered two possible solutions:

1. Sort the entries by objectSid and use filters to get all the objects. Sample Code
   I don't like that for several reasons:
   • It seems unpredictable to mess with the objectSid, as you have to take it apart, convert it to decimal, convert it back ...
   • I don't see how you can compare these id's.
     (I've tried: '&((objectClass=user)(objectSid>=0))')
     
     
2. Filter after the first letters of the object names (as suggested here):
   That's not an optimal solution as many of the users/groups in our system are prefixed with the same few letters.

So my question:

What approach is best used here?
If it's the first one, how can I be sure to handle the objectSid correctly?

Any other possibilities? Am I missing something obvious?

Update:
- This related question provides information about why the Simple Paged Results extension does not work.
- The web server is running on a Linux server, so COM objects/adoDB are not an option.

Answer 1

As I've not found any clean solutions I decided to go with the first approach: Filtering By Object-Sids.

This workaround has it's limitations:

1. It only works for objects with an objectsid, i.e Users and Groups.
2. It assumes that all Users/Groups are created by the same authority.
3. It assumes that there are not more missing relative SIDs than the size limit.

The idea is it to first read all possible objects and pick out the one with the lowest relative SID. The relative SID is the last chunk in the SID:

S-1-5-21-3188256696-111411151-3922474875-1158

Let's assume this is the lowest relative SID in a search that only returned 'Partial Search Results'. Let's further assume the size limit is 1000.

The program then does the following: It searches all Objects with the SIDs between

S-1-5-21-3188256696-111411151-3922474875-1158
and
S-1-5-21-3188256696-111411151-3922474875-0159

then all between

S-1-5-21-3188256696-111411151-3922474875-1158
and
S-1-5-21-3188256696-111411151-3922474875-2157

and so on until one of the searches returns zero objects.

There are several problems with this approach, but it's sufficient for my purposes.
The Code:

$filter = '(objectClass=Group)';
$attributes = array('objectsid','cn'); //objectsid needs to be set

$result = array();

$maxPageSize = 1000;
$searchStep = $maxPageSize-1;

$adResult = @$adConn->search($filter,$attributes); //Supress warning for first query (because it exceeds the size limit)

//Read smallest RID from the resultset
$minGroupRID = '';

for($i=0;$i<$adResult['count'];$i++){
    $groupRID = unpack('V',substr($adResult[$i]['objectsid'][0],24));
    if($minGroupRID == '' || $minGroupRID>$groupRID[1]){
        $minGroupRID = $groupRID[1];
    }    
}

$sidPrefix =  substr($adResult[$i-1]['objectsid'][0],0,24);   //Read last objectsid and cut off the prefix

$nextStepGroupRID = $minGroupRID;

do{ //Search for all objects with a lower objectsid than minGroupRID
    $adResult = $adConn->search('(&'.$filter.'(objectsid<='.preg_replace('/../','\\\\$0',bin2hex($sidPrefix.pack('V',$nextStepGroupRID))).')(objectsid>='.preg_replace('/../','\\\\$0',bin2hex($sidPrefix.pack('V',$nextStepGroupRID-$searchStep))).'))', $attributes);

    for($i=0;$i<$adResult['count'];$i++){
        $RID = unpack('V',substr($adResult[$i]['objectsid'][0],24));    //Extract the relative SID from the SID
        $RIDs[] = $RID[1];

        $resultSet = array();
        foreach($attributes as $attribute){
            $resultSet[$attribute] = $adResult[$i][$attribute][0];
        }
        $result[$RID[1]] = $resultSet;
    }

    $nextStepGroupRID = $nextStepGroupRID-$searchStep;

}while($adResult['count']>1);

$nextStepGroupRID = $minGroupRID;

do{ //Search for all object with a higher objectsid than minGroupRID  
    $adResult = $adConn->search('(&'.$filter.'(objectsid>='.preg_replace('/../','\\\\$0',bin2hex($sidPrefix.pack('V',$nextStepGroupRID))).')(objectsid<='.preg_replace('/../','\\\\$0',bin2hex($sidPrefix.pack('V',$nextStepGroupRID+$searchStep))).'))', $attributes);

    for($i=0;$i<$adResult['count'];$i++){
        $RID = unpack('V',substr($adResult[$i]['objectsid'][0],24));    //Extract the relative SID from the SID
        $RIDs[] = $RID[1];

        $resultSet = array();
        foreach($attributes as $attribute){
            $resultSet[$attribute] = $adResult[$i][$attribute][0];
        }
        $result[$RID[1]] = $resultSet;
    }

    $nextStepGroupRID = $nextStepGroupRID+$searchStep;

}while($adResult['count']>1);

var_dump($result);

The $adConn->search method looks like this:

function search($filter, $attributes = false, $base_dn = null) {
        if(!isset($base_dn)){
            $base_dn = $this->baseDN;
        }

        $entries = false;
        if (is_string($filter) && $this->bind) {
                if (is_array($attributes)) {
                        $search  = ldap_search($this->resource, $base_dn, $filter, $attributes);
                } else {
                        $search  = ldap_search($this->resource, $base_dn, $filter);
                }
                if ($search !== false) {
                        $entries = ldap_get_entries($this->resource, $search);
                }
        }
        return $entries;
}