Whats the difference

[pueraeternus]

I think the reason is that the dhyanas were first elevated to a very high position and accordingly their definitions changed. Even today there are a variety of teachers with different views regarding what the dhyanas actually stand for (especially among Theravada masters). Since it became so removed from actual practice, it was for the better that later teachers simply dismissed them as unnecessary or even misguided.

Oh yes - excellent point. The Visuddhimagga gave such an arduous program and definition of what constituted real dhyana, that most people would never attain it.

I am not sure how or if the other Sravakayana schools denote dhyana that way - it seems closer to what the Agamas taught, than how it is described in the Visuddhimagga. From my recollection, the Sravakabhumi's description of dhyana practice is not that arduous, and likewise the other Abhidharma texts I studied before. Seems quite reasonable.

[Astus]

Here is a summary from Tsele Natsok's Heart Lamp (p. 31-33) on the nine dhyanas. Interestingly, it is not under the section discussing shamatha but the chapter about "Faults and Qualities", sub-chapter "Clearing Away Specific Errors and Mistakes". The reason given is the fault of attaching to bliss, clarity and nonthought. What he originally defines as the proper practice of shamatha in Mahamudra is resting naturally without fixating on thoughts or appearances. So, his description of the dhyanas:

When resting evenly in shamatha, then, to be free from gross thoughts of perceiver and perceived, but still to be fettered by the concept of meditator and meditation object is called the "samadhi of the First Dhyana." Why is that? Because this is what is being meditated upon in all the abodes of the gods of the First Dhyana. Meditating in this way creates the cause for being reborn there as a god at the level of the First Dhyana.
Likewise, the second dhyana is to be free from the state of mind of concept and discernment, but still to experience the taste of the samadhi of joy and bliss.
The third dhyana is to be free from mental movement, but supported merely by the inhalation and exhalation of breath.
The fourth dhyana is to be free from all kinds of thoughts, a state of samadhi which is unobstructed clarity, like space.
Supreme among all the mundane samadhis, these are the foundation for vipashyana. If meditated upon with attachment, however, they become a deviation from Mahamudra causing rebirth as gods in the abodes of those dhyanas.
Furthermore, thinking "All phenomena are infinite like space!" or "This consciousness, free from partiality and nonexistent, is infinite!" or "Perception, being neither existent nor nonexistent, is not an action of mind!" or "This mind is voidness, which is nothing whatsoever!"Dwelling in the states of these four levels has the defect of straying into the four formless spheres of finality; called the Infinite Space, Infinite Consciousness, Neither Presence nor Absence, and Nothing Whatsoever.
The shravaka's samadhi of peace is the state of mind that has abandoned these four thoughts in which involvement in objects has been blocked, and in which you abide having interrupted the movements of the wind-mind. Although such a state is taught to be the ultimate shamatha, in this context it is not a faultless meditation unless embraced by vipashyana.
Each of these nine dhyanas of absorption has some temporary qualities, such as accomplishment of superknowledges and miraculous powers. Here, however, you should attain the ultimate result of complete enlightenment and not merely relative or superficial qualities. Thus, if these are accomplished naturally and you then cling to them or feel arrogant, know that to be a direct obstacle for enlightenment.

[monktastic]

What he originally defines as the proper practice of shamatha in Mahamudra is resting naturally without fixating on thoughts or appearances.

Which brings up another fascinating point! Back when I was doing shamatha retreat 4 years ago, I could not reconcile Alan Wallace's suggestion to treat shamatha as one-pointedness with the Mahamudra / Dzogchen approach. I can look for a quote, but somewhere Tsoknyi Rinpoche says something like "focus on the object with 80% attention, and let 20% be aware of surroundings."

This year, I heard a Wallace lecture where he admits that he heard this for the first time only recently, and that he considers it quite a departure from Tsongkhapa (and hence the Visuddhimagga). He thinks attaining full dhyana (or the access to it, just prior to attainment of it) has historically been considered necessary. As he greatly respects the modern teachers, he doesn't think they are wrong but thinks it is an open question as to whether this approach will work.

(Certainly the shamatha that is merged with vipashyana cannot be fixated, so I think he is referring to the preliminary shamatha.)

[Astus]

Regarding not fully focusing on the object:

"The importance of not focusing on the meditative object too intently should be emphasized here. You need to restrain your mind from wandering away from the meditation object, but you should also maintain a sense of relaxation. ... The function of the meditative object is to enhance your concentration. Simply be aware of the object, rather than grasping onto it or trying to focus on it too intently. An element of relaxation is extremely important here. You should not mistake the object of meditation for the function of meditation. The function of meditation is not to concentrate on a piece of wood or any other object."
(Traleg Kyabgon: Ocean of Certainty, p. 80)

"What is very important, the masters always advise, is not to fixate while practising the concentration of calm abiding. That's why they recommend you place only twenty five percent of the attention on mindfulness of the breath. But then, as you may have noticed, mindfulness alone is not enough. While you are supposed to be watching the breath, after a few minutes you may find yourself playing in a football match or starring in your own film. So another twenty five percent should be devoted to a continuous and watchful awareness, one that oversees and checks whether you are being mindful of the breath. The remaining fifty percent of your attention is left abiding, spaciously. Of course the exact percentages are not as important as the fact that all three of these elements-mindfulness, awareness and spaciousness are present."
(Sögyal Rinpoche: Natural Great Peace, p. 7)

[pueraeternus]

So, his description of the dhyanas:

... the four formless spheres of finality; called the Infinite Space, Infinite Consciousness, Neither Presence nor Absence, and Nothing Whatsoever.

This is just a technicality. but I am surprised to see the sequence of the last 2 arupya samapattis flipped - it ought to be Nothing Whatsover (akimcanyayatana), then followed by Neither Presence nor Abscence (Naivasamjnanasamjnayatana).

[Astus]

I've seen that order before, can't remember where. But it is somewhat logical.

[Jnana]

This year, I heard a Wallace lecture where he admits that he heard this for the first time only recently, and that he considers it quite a departure from Tsongkhapa (and hence the Visuddhimagga). He thinks attaining full dhyana (or the access to it, just prior to attainment of it) has historically been considered necessary. As he greatly respects the modern teachers, he doesn't think they are wrong but thinks it is an open question as to whether this approach will work.

With sūtra mahāmudrā one is introduced to the view after having established some degree of śamatha. Kagyu teachers usually consider the ninth stage of mental abiding of the desire realm to be a sufficient degree of calm. Dakpo Tashi Namgyal's Moonlight:

How then should one seek to realize tranquility? It is highly praiseworthy for someone to achieve simple tranquility on the preparatory level of the first concentrative absorption [on the plane of sublime form], as stated before. Failing that, one would do well to realize a one-pointed concentration on the plane of desire.

And:

If, during this quiescent state, the mind unceasingly perceives forms, sounds, and the other senses gently, serenely, fearlessly, vividly, and quietly, tranquility has been achieved.

This differs from Je Tsongkhapa who maintained that the preparatory level of the first dhyāna was necessary. But generally, mahāmudrā and dzogchen are mindfulness based practice traditions, and with essence mahāmudrā and dzogchen one is introduced to the view and then maintains non-distraction as the meditation. Tsele Natsok Rangdröl, The Circle of the Sun:

Concerning mindfulness serving or not serving as the meditation: some deluded people appear to concentrate with rigid fixation and believe that keeping their mind hostage is the meditation of mahāmudrā. That is nothing but their personal fault. The authentic great Kagyu masters took self-cognizant mindfulness as their practice, which is identical to the primordially pure self-awareness of the dzogchen system. Thus, despite different terminology, there is no difference in meaning. Neither system, mahāmudrā nor dzogchen, considers that meditation is the conceptual mind that fixates on mindfulness.

The reason for this is succinctly stated by Tulku Urgyen Rinpoche, As It Is, Vol. II:

There is no need to do anything to your present wakefulness at that moment; it is already as it is. That is the true meaning of naked ordinary mind, tamal kyi shepa, a famous term in Tibetan. Ordinary mind means not tampered with. There is no 'thing' there which needs to be accepted or rejected; it is simply as it is.

[pueraeternus]

Good to see you again Jnana!