I'd go down a draconian path to influence behaviour so that potential future crims would choose a different path because they'd fear the consequences of their criminality to a much greater degree than they do presently. First up would be to make the prison experience for convicted felons akin to what it was in the early Victorian era which would elbow the concept of rehabilitation to concentrate on punishment. That would include all single cells, no association with one another and absolute silence at all times. There'd me no work activities other than that of mindless grind designed to break their spirit... shovelling tons of sand back and forth from A to B for 8 hours per day and the like. Only those of State Pension age and others adjudged unfit by a doctor would be excused.

Introduce whole life sentences for all murderers and minimum term tariffs for a long and wide variety of serious offences. That way a home burglar, for example, with 5 years to serve had better behave impeccably lest he'd have no prospect of release after serving the minimum. Each establishment would have a dedicated wing for remand inmates where they'd have to comply with a far more relaxed regime. Mental defectives would be placed in secure hospitals as they are today.

A far cheaper option would be the Saudi route. Murder, peddle drugs or commit rape there and it's goodnight Vienna. Thieves lose a hand. Unsurprisingly those crimes are ultra rare.