I see no reason to sign a session cookie that was issued by and will be validated against some authentication service. Especially if the cookie is sourced from a cryptographically-secure pseudorandom sequence of bytes which is sufficiently long to avoid collisions with other such sequences (e.g. 256 bits of entropy should typically do per session cookie).
In my implementations of such schemes, I will typically feed the CSPRNG output with a time-based salt into SHA256 or SHA512 in order to generate the final token, which is simply stored on both ends (user session db row on server & cookie on client). At this point, the server can simply 1:1 the supplied client token with the row in the database to determine if the session is valid and which claims to return (or which validated final server response to return as the case may be).
Advantages with using a token made out of pure entropy soup are that it's consistent throughout the entire user's session, and confers absolutely zero information regarding the nature of that session. Disadvantages are honestly unclear. I feel like JWT and other stateless schemes started as a bandaid to re-couple the authentication domains of disparate business services, but by way of the various clients.
This property you use is known as an ephemeral session key and is a requirement for perfect forward secrecy. (Not being able to crack previous messages by compromising a key in the future)
You can store data in the server-side session the user is never intended to see. If you want to store that in a JWT, you're going to have to encrypt it, not just sign, and even that may clue them in that something's going on - if you perform some potentially spammy act on a site and suddenly the JWT gets updated with a big encrypted block, you probably know you tripped some flag.