THE Castle home would be rebuilt at Mayday Hills in Beechworth but not open to tourists under revised plans.
The fresh approach by owner George Fendyk follows Indigo Shire councillors on Tuesday night voting down his plan for a caravan park at the old asylum site.
The weatherboard house from the 1997 movie was going to form the office for the caravan park.
Mr Fendyk, who described the council's rejection of his dream as a "stupid decision", said he would not appeal it to planning umpire VCAT.
"I don't think it's worthwhile appealing to any bastard really," Mr Fendyk said.
He said he aimed to re-assemble the house, which was trucked from Melbourne's northern suburbs in December 2017, and put in a planning application for council approval.
Visitors will be able to look at its exterior and have photos with it as a backdrop, but not enter it as originally envisaged.
"People can drive around the outside of it," Mr Fendyk said.
"You can't man it, it wouldn't make enough money to maintain it.
"I was going to have it as an office for the caravan park, but you can't pay for someone to open it up to the public.
"It ain't going to work."
Councillors voted 5-2 against the caravan park plan, despite staff recommending it to be approved subject to conditions.
Former mayor Jenny O'Connor said the proposed cabins were not in keeping with the site's heritage status and it would lead to "a ghetto situation".
"There's a lot of concern from the community about that and I'm responding to that concern," she said.
The council's planning and statutory services manager Ian Scholes noted the site earmarked for the caravan park was separated from homes and the "historic core" of Mayday Hills.
"The subject land appears to be an appropriate site for the proposed use," he determined.
Four objections to the caravan park were received.
Concerns were centred on noise, additional traffic and a caravan park not being compatible with the land being zoned rural.
Mayor Bernard Gaffney spoke against voting down the caravan park plan and questioned why Cr O'Connor, a fellow Beechworth resident, was against it based on the vibe.
He said the conditions for approval were "quite binding and I don't think they would effect the amenity or the vibe of the area".
Cr Gaffney told The Border Mail on Wednesday that the re-assembly of the house would require a permit, but given it was not a major decision it may be issued by officers rather than be subject to council debate.