Is it possible to ask agents to leave an agentset without relying on their variables in NetLogo?

Question

Although NetLogo has a good set of primitives allowing for the handy creation of agentsets, sometimes we might want to ask agents to 'manually' add themselves to an agentset as part of a procedure they're running - for instance, by using turtle-set.

Example:

globals [
 good-folks
 bad-folks 
]

to setup
  clear-all
  
  set good-folks (turtle-set)
  set bad-folks (turtle-set)
  
  create-turtles 50 [
   ifelse (random 2 < 1)
    [set good-folks (turtle-set good-folks self)]
    [set bad-folks (turtle-set bad-folks self)] 
  ]
end

to go
  ask good-folks [
   do-good-things 
  ]
  
  ask bad-folks [
   do-bad-things 
  ]
end

to do-good-things end
to do-bad-things end

The advantage of this approach above is better readability, the possibility to avoid having to create a turtles-own variable and avoid having to resort to with or if statements - as shown below:

turtles-own [
 group 
]

to setup
  clear-all
  
  create-turtles 50 [
   ifelse (random 2 < 1)
    [set group "good folks"]
    [set group "bad folks"]
  ]
end

to go1
  ask turtles with [group = "good folks"] [
   do-good-things 
  ]
  
  ask turtles with [group = "bad folks"] [
   do-bad-things 
  ]
end

; OR

to go2
  ask turtles [
   ifelse (group = "good folks")
    [do-good-things]
    [do-bad-things]
  ]
end

to do-good-things end
to do-bad-things end

Especially, it seems to me that the latter approach does not scale up as smoothly as the former, i.e. more groups will require more withs or ifs, instead of just being able to refer to the needed group neatly.

However, I am not aware of a way to do the exact opposite of set good-folks (turtle-set good-folks self), i.e. asking the turtle to 'manually' remove itself from the agentset.

; In this case, bad-folks try to recruit some good folks
to do-bad-things
  if (random 3 < 2) [
   ask one-of good-folks [
     set good-folks (turtle-set good-folks - self) ; Leave the good folks <- **How to do this?**
     set bad-folks (turtle-set bad-folks self) ; Join the bad folks
    ] 
  ]
end

I know this could be achieved by using a turtles-own variable and then updating agentsets as set <good/bad>-folks turtles with [...], but this is exactly the whole thing that I am trying to avoid in this question: less readable, more variables, more commands etc. And, perhaps equally important, it looks inconsistent with the use that can be done of turtle-set to grow the agentset by using it in combination with self. It would look easier and more consistent if existed a way to subtract self from the agentset - does it exist?

Update

Jasper's solution (i.e. using breeds instead of agentsets) works as long as the groups that I want to keep track of are mutually exclusive: agents switch from one single breed to another single breed only needing set breed ..., which will both make the agent join the new breed and leave their previous breed. However, this would not work in case I want to keep track of multiple groups with overlaps: beyond good-folks and bad-folks, I may have tall-folks and short-folks. Agents will have to belong to multiple groups simultaneously - something that agentsets can handle while breeds cannot.

Answer 1

The answer, which I realised following some line of thought after Jasper's reply, is actually quite simple and it only requires other:

to do-bad-things
  if (random 3 < 2) [
   ask one-of good-folks [
     set good-folks (turtle-set other good-folks) ; Leave the good folks
     set bad-folks (turtle-set bad-folks self) ; Join the bad folks
    ] 
  ]
end

Extra cheap thought

This makes me also think that it would be handy if NetLogo had two primitives called join and leave that worked like:

to join [agentset]
  (ifelse
    is-turtle? self [
     set agentset (turtle-set agentset self)
    ]
    is-patch? self [
     set agentset (patch-set agentset self)
    ]
    ; else command:
    [set agentset (link-set agentset self)])
end

to leave [agentset]
  (ifelse
    is-turtle? self [
     set agentset (turtle-set other agentset)
    ]
    is-patch? self [
     set agentset (patch-set other agentset)
    ]
    ; else command:
    [set agentset (link-set other agentset)])
end

So that one could do:

to do-bad-things
  if (random 3 < 2) [
   ask one-of good-folks [
     leave good-folks
     join bad-folks
    ] 
  ]
end

Which feels almost like reading a book rather than reading code; although I recognise the need for it in the end is really minimal.