Christ said:

John 3:3 Jesus answered and said unto him, Verily, verily, I say unto thee, Except a man be born again, he cannot see the kingdom of God.

Paul said in his second epistle to Corinth:

2 Corinthians 5:17 Therefore if any man be in Christ, he is a new creature: old things are passed away; behold, all things are become new.

Calvin’s commentary on this verse explains what is meant:

John Calvin’s Verse Commentary
2 Corinthians 5:17

17. Therefore if any man is in Christ. As there is something wanting in this expression, it must be supplied in this way — “If any one is desirous to hold some place in Christ, that is, in the kingdom of Christ, or in the Church 1 let him be a new creature ” By this expression he condemns every kind of excellence that is wont to be in much esteem among men, if renovation of heart is wanting. “Learning, it is true, and eloquence, and other endowments, are valuable, and worthy to be honored; but, where the fear of the Lord and an upright conscience are wanting, all the honor of them goes for nothing. Let no one, therefore, glory in any distinction, inasmuch as the chief praise of Christians is self-renunciation.”

Nor is this said merely for the purpose of repressing the vanity of the false apostles, but also with the view of correcting the ambitious judgments of the Corinthians, in which outward disguises were of more value than real sincerity — though this is a fault that is common to almost all ages. For where shall we find the man that does not attach much more importance to show, than to true holiness? Let us, therefore, keep in view this admonition — that all that are not renewed by the Spirit of God, should be looked upon as nothing in the Church, by whatever ornaments they may in other respects be distinguished.

Old things are passed away. When the Prophets speak of the kingdom of Christ, they foretell that there will be new heavens and a new earth, (Isa 65:17,) meaning thereby, that all things will be changed for the better, until the happiness of the pious is completed. As, however, Christ’s kingdom is spiritual, this change must take place chiefly in the Spirit, and hence it is with propriety that he begins with this. There is, therefore, an elegant and appropriate allusion, when Paul makes use of a commendation of this kind, for the purpose of setting forth the value of regeneration. Now by old things he means, the things that are not formed anew by the Spirit of God. Hence this term is placed in contrast with renewing grace. The expression passed away, he uses in the sense of fading away, as things that are of short duration are wont to fall off, when they have passed their proper season. Hence it is only the new man, that flourishes and is vigorous 2 in the kingdom of Christ.

Outward conformity towards things of God, where before there was indifference or disdain, does not describe a real new creation. A turning away from all things God hates, that formerly was part of our lives–what God hates, we hate; what God loves, we now love.

In his first epistle to Corinth, Paul lists just a few things that unless turned away from, one will not enter the Kingdom of Heaven, for all one’s profession. Faith is not merely an outward profession, but a way of life–embraced and loved by the new creature the Christian is, because Christ loves those things; and the things we formerly took part in, pass away, us now by the possession of God’s Spirit, having a hatred for sin, especially our own.
God does not call his people to a way of life and leave them helpless and impotent to live up to our calling. The regeneration, wrought by the Holy Spirit, empowers us to live up to our calling, and have the power of godliness to conqueor sin. The Hoy Spirit is the seal upon our salvation; as for all outward formal profession–those without God’s Spirit dwelling in them will not be new creations, the old things will not pass away. We maybe able to give a credible profession for an hour on Sunday mornings before men but none of us can hide from or fool God, there is no hiding from him what our real estate be; at home, at work, at play, his all-seeing, all-searching eye is upon us, he knows us better than we know ourselves;

1 Corinthians 5:11 But now I have written unto you not to keep company, if any man that is called a brother be a fornicator, or covetous, or an idolater, or a railer, or a drunkard, or an extortioner; with such an one no not to eat.

Secondly, the Holy Spirit is the Author of regeneration. Man by nature is spiritually dead and separated from God, being completely immersed in the things of the flesh. He is as full of sin as a dead body is full of worms. When the moment of good pleasure arrives for each of the elect, however, the Holy Spirit quickens and grants him spiritual life, this being the consequence of the soul’s union with God in Christ. As a result of this, Christ is formed in them and the spiritual frame of their soul inclines towards Jesus. That which previously was so desirable to their eye has now become despicable. That in which they previously delighted now causes sorrow. Those activities they formerly sought out, they now flee. Their mind, will, and affections have been changed. They have become new creatures, and in consequence of this change wrought within the soul, thoughts concerning God and reflections upon heavenly things become prevalent. All this results in a different manner of speech, in godly conversation, in holiness of life, in having a delight in the godly, in dignified behavior, as well as in modest dress. In a word, this change can be compared to a dead person arising from the grave. In its initial manifestation, however, this new life has many imperfections. In its beginning it is feeble and grows slowly, which is also true for its external manifestation. It is all only in part, but nevertheless in truth. It is this life which the Holy Spirit works, “the Spirit giveth life” (2 Cor. 3:6); “and renewing of the Holy Ghost” (Titus 3:5); “Except a man be born of water and of the Spirit, he cannot enter into the kingdom of God” (John 3:5).

Even though the Holy Spirit could accomplish this without means, it pleases Him to use the Word as a means. Nevertheless He immediately (that is, without means) touches the soul in a manner not known to us, exerting a creative power similar as at the time of creation when He moved upon the face of the waters. The Hebrew uses the word מְרַחֶפֶת  (Merachepheth), which is indicative of motion that forms and brings forth. As I stated, the Spirit uses the Word in regeneration. “Of His own will begat He us with the Word of truth” (James 1:18); “Being born again, not of corruptible seed, but of incorruptible, by the Word of God, which liveth and abideth for ever” (1 Pet. 1:23).
—Wilhelmus â Brakel “The Christian’s Reasonable Service”