Genesis 40:4

40:4 The captain of the guard appointed Joseph to be their attendant, and he served them.

They spent some time in custody.