Hi Marc sorry for the slow reply.
OAuth is supported with Extranet users. I just discussed this with a few people and it should work.
It feels like your code is ok since you were able to get an access token for an internal user.
When the internal user logs in, are they getting to their proper SAP Jam external user branded page?
We are looking at the error message to see if that might give us a hint.