The word " mass" has two meanings in special relativity: invariant mass (also called rest mass) is an invariant quantity which is the same for all observers in all reference frames, while the relativistic mass is dependent on the velocity of the observer. According to the concept of mass–energy equivalence, invariant mass is equivalent to rest energy, while relativistic mass is equivalent to relativistic energy (also called total energy).
The term "relativistic mass" tends not to be used in particle and nuclear physics and is often avoided by writers on special relativity, in favor of referring to the body's relativistic energy.  In contrast, "invariant mass" is usually preferred over rest energy. The measurable inertia and the warping of spacetime by a body in a given frame of reference is determined by its relativistic mass, not merely its invariant mass. For example, photons have zero rest mass but contribute to the inertia (and weight in a gravitational field) of any system containing them.
The concept is generalized in mass in general relativity.
The term mass in special relativity usually refers to the rest mass of the object, which is the Newtonian mass as measured by an observer moving along with the object. The invariant mass is another name for the rest mass of single particles. The more general invariant mass (calculated with a more complicated formula) loosely corresponds to the "rest mass" of a "system". Thus, invariant mass is a natural unit of mass used for systems which are being viewed from their center of momentum frame (COM frame), as when any closed system (for example a bottle of hot gas) is weighed, which requires that the measurement be taken in the center of momentum frame where the system has no net momentum. Under such circumstances the invariant mass is equal to the relativistic mass (discussed below), which is the total energy of the system divided by c2 (the speed of light squared).
The concept of invariant mass does not require bound systems of particles, however. As such, it may also be applied to systems of unbound particles in high-speed relative motion. Because of this, it is often employed in particle physics for systems which consist of widely separated high-energy particles. If such systems were derived from a single particle, then the calculation of the invariant mass of such systems, which is a never-changing quantity, will provide the rest mass of the parent particle (because it is conserved over time).
It is often convenient in calculation that the invariant mass of a system is the total energy of the system (divided by c2) in the COM frame (where, by definition, the momentum of the system is zero). However, since the invariant mass of any system is also the same quantity in all inertial frames, it is a quantity often calculated from the total energy in the COM frame, then used to calculate system energies and momenta in other frames where the momenta are not zero, and the system total energy will necessarily be a different quantity than in the COM frame. As with energy and momentum, the invariant mass of a system cannot be destroyed or changed, and it is thus conserved, so long as the system is closed to all influences. (The technical term is isolated system meaning that an idealized boundary is drawn around the system, and no mass/energy is allowed across it.)
The relativistic mass is the sum total quantity of energy in a body or system (divided by c2). Thus, the mass in the formula
In the center of momentum frame, and the relativistic mass equals the rest mass. In other frames, the relativistic mass (of a body or system of bodies) includes a contribution from the "net" kinetic energy of the body (the kinetic energy of the center of mass of the body), and is larger the faster the body moves. Thus, unlike the invariant mass, the relativistic mass depends on the observer's frame of reference. However, for given single frames of reference and for isolated systems, the relativistic mass is also a conserved quantity. The relativistic mass is also the proportionality factor between velocity and momentum,
Newton's second law remains valid in the form
When a body emits light of frequency and wavelength as a photon of energy , the mass of the body decreases by ,  which some   interpret as the relativistic mass of the emitted photon since it also fulfills . Although some authors present relativistic mass as a fundamental concept of the theory, it has been argued that this is wrong as the fundamentals of the theory relate to space–time. There is disagreement over whether the concept is pedagogically useful.    It explains simply and quantitatively why a body subject to a constant acceleration cannot reach the speed of light, and why the mass of a system emitting a photon decreases.  In relativistic quantum chemistry, relativistic mass is used to explain electron orbital contraction in heavy elements.   The notion of mass as a property of an object from Newtonian mechanics does not bear a precise relationship to the concept in relativity.  Relativistic mass is not referenced in nuclear and particle physics,  and a survey of introductory textbooks in 2005 showed that only 5 of 24 texts used the concept,  although it is still prevalent in popularizations.
If a stationary box contains many particles, it weighs more in its rest frame, the faster the particles are moving. Any energy in the box (including the kinetic energy of the particles) adds to the mass, so that the relative motion of the particles contributes to the mass of the box. But if the box itself is moving (its center of mass is moving), there remains the question of whether the kinetic energy of the overall motion should be included in the mass of the system. The invariant mass is calculated excluding the kinetic energy of the system as a whole (calculated using the single velocity of the box, which is to say the velocity of the box's center of mass), while the relativistic mass is calculated including invariant mass plus the kinetic energy of the system which is calculated from the velocity of the center of mass.
Relativistic mass and rest mass are both traditional concepts in physics, but the relativistic mass corresponds to the total energy. The relativistic mass is the mass of the system as it would be measured on a scale, but in some cases (such as the box above) this fact remains true only because the system on average must be at rest to be weighed (it must have zero net momentum, which is to say, the measurement is in its center of momentum frame). For example, if an electron in a cyclotron is moving in circles with a relativistic velocity, the mass of the cyclotron+electron system is increased by the relativistic mass of the electron, not by the electron's rest mass. But the same is also true of any closed system, such as an electron-and-box, if the electron bounces at high speed inside the box. It is only the lack of total momentum in the system (the system momenta sum to zero) which allows the kinetic energy of the electron to be "weighed". If the electron is stopped and weighed, or the scale were somehow sent after it, it would not be moving with respect to the scale, and again the relativistic and rest masses would be the same for the single electron (and would be smaller). In general, relativistic and rest masses are equal only in systems which have no net momentum and the system center of mass is at rest; otherwise they may be different.
The invariant mass is proportional to the value of the total energy in one reference frame, the frame where the object as a whole is at rest (as defined below in terms of center of mass). This is why the invariant mass is the same as the rest mass for single particles. However, the invariant mass also represents the measured mass when the center of mass is at rest for systems of many particles. This special frame where this occurs is also called the center of momentum frame, and is defined as the inertial frame in which the center of mass of the object is at rest (another way of stating this is that it is the frame in which the momenta of the system's parts add to zero). For compound objects (made of many smaller objects, some of which may be moving) and sets of unbound objects (some of which may also be moving), only the center of mass of the system is required to be at rest, for the object's relativistic mass to be equal to its rest mass.
A so-called massless particle (such as a photon, or a theoretical graviton) moves at the speed of light in every frame of reference. In this case there is no transformation that will bring the particle to rest. The total energy of such particles becomes smaller and smaller in frames which move faster and faster in the same direction. As such, they have no rest mass, because they can never be measured in a frame where they are at rest. This property of having no rest mass is what causes these particles to be termed "massless". However, even massless particles have a relativistic mass, which varies with their observed energy in various frames of reference.
The invariant mass is the ratio of four-momentum (the four-dimensional generalization of classical momentum) to four-velocity: 
The relativistic expressions for E and p obey the relativistic energy–momentum relation: 
The equation is also valid for photons, which have m = 0:
A photon's momentum is a function of its energy, but it is not proportional to the velocity, which is always c.
For an object at rest, the momentum p is zero, therefore
The rest mass is only proportional to the total energy in the rest frame of the object.
When the object is moving, the total energy is given by
To find the form of the momentum and energy as a function of velocity, it can be noted that the four-velocity, which is proportional to , is the only four-vector associated with the particle's motion, so that if there is a conserved four-momentum , it must be proportional to this vector. This allows expressing the ratio of energy to momentum as
This results in
these expressions can be written as
When working in units where c = 1, known as the natural unit system, all the relativistic equations are simplified and the quantities energy, momentum, and mass have the same natural dimension: 
The equation is often written this way because the difference is the relativistic length of the energy momentum four-vector, a length which is associated with rest mass or invariant mass in systems. Where m > 0 and p = 0, this equation again expresses the mass–energy equivalence E = m.
The rest mass of a composite system is not the sum of the rest masses of the parts, unless all the parts are at rest. The total mass of a composite system includes the kinetic energy and field energy in the system.
The total energy E of a composite system can be determined by adding together the sum of the energies of its components. The total momentum of the system, a vector quantity, can also be computed by adding together the momenta of all its components. Given the total energy E and the length (magnitude) p of the total momentum vector , the invariant mass is given by:
In the system of natural units where c = 1, for systems of particles (whether bound or unbound) the total system invariant mass is given equivalently by the following:
Where, again, the particle momenta are first summed as vectors, and then the square of their resulting total magnitude ( Euclidean norm) is used. This results in a scalar number, which is subtracted from the scalar value of the square of the total energy.
For such a system, in the special center of momentum frame where momenta sum to zero, again the system mass (called the invariant mass) corresponds to the total system energy or, in units where c = 1, is identical to it. This invariant mass for a system remains the same quantity in any inertial frame, although the system total energy and total momentum are functions of the particular inertial frame which is chosen, and will vary in such a way between inertial frames as to keep the invariant mass the same for all observers. Invariant mass thus functions for systems of particles in the same capacity as "rest mass" does for single particles.
Note that the invariant mass of an isolated system (i.e., one closed to both mass and energy) is also independent of observer or inertial frame, and is a constant, conserved quantity for isolated systems and single observers, even during chemical and nuclear reactions. The concept of invariant mass is widely used in particle physics, because the invariant mass of a particle's decay products is equal to its rest mass. This is used to make measurements of the mass of particles like the Z boson or the top quark.
Total energy is an additive conserved quantity (for single observers) in systems and in reactions between particles, but rest mass (in the sense of being a sum of particle rest masses) may not be conserved through an event in which rest masses of particles are converted to other types of energy, such as kinetic energy. Finding the sum of individual particle rest masses would require multiple observers, one for each particle rest inertial frame, and these observers ignore individual particle kinetic energy. Conservation laws require a single observer and a single inertial frame.
In general, for isolated systems and single observers, relativistic mass is conserved (each observer sees it constant over time), but is not invariant (that is, different observers see different values). Invariant mass, however, is both conserved and invariant (all single observers see the same value, which does not change over time).
The relativistic mass corresponds to the energy, so conservation of energy automatically means that relativistic mass is conserved for any given observer and inertial frame. However, this quantity, like the total energy of a particle, is not invariant. This means that, even though it is conserved for any observer during a reaction, its absolute value will change with the frame of the observer, and for different observers in different frames.
By contrast, the rest mass and invariant masses of systems and particles are both conserved and also invariant. For example: A closed container of gas (closed to energy as well) has a system "rest mass" in the sense that it can be weighed on a resting scale, even while it contains moving components. This mass is the invariant mass, which is equal to the total relativistic energy of the container (including the kinetic energy of the gas) only when it is measured in the center of momentum frame. Just as is the case for single particles, the calculated "rest mass" of such a container of gas does not change when it is in motion, although its "relativistic mass" does change.
The container may even be subjected to a force which gives it an overall velocity, or else (equivalently) it may be viewed from an inertial frame in which it has an overall velocity (that is, technically, a frame in which its center of mass has a velocity). In this case, its total relativistic mass and energy increase. However, in such a situation, although the container's total relativistic energy and total momentum increase, these energy and momentum increases subtract out in the invariant mass definition, so that the moving container's invariant mass will be calculated as the same value as if it were measured at rest, on a scale.
All conservation laws in special relativity (for energy, mass, and momentum) require isolated systems, meaning systems that are totally isolated, with no mass–energy allowed in or out, over time. If a system is isolated, then both total energy and total momentum in the system are conserved over time for any observer in any single inertial frame, though their absolute values will vary, according to different observers in different inertial frames. The invariant mass of the system is also conserved, but does not change with different observers. This is also the familiar situation with single particles: all observers calculate the same particle rest mass (a special case of the invariant mass) no matter how they move (what inertial frame they choose), but different observers see different total energies and momenta for the same particle.
Conservation of invariant mass also requires the system to be enclosed so that no heat and radiation (and thus invariant mass) can escape. As in the example above, a physically enclosed or bound system does not need to be completely isolated from external forces for its mass to remain constant, because for bound systems these merely act to change the inertial frame of the system or the observer. Though such actions may change the total energy or momentum of the bound system, these two changes cancel, so that there is no change in the system's invariant mass. This is just the same result as with single particles: their calculated rest mass also remains constant no matter how fast they move, or how fast an observer sees them move.
On the other hand, for systems which are unbound, the "closure" of the system may be enforced by an idealized surface, inasmuch as no mass–energy can be allowed into or out of the test-volume over time, if conservation of system invariant mass is to hold during that time. If a force is allowed to act on (do work on) only one part of such an unbound system, this is equivalent to allowing energy into or out of the system, and the condition of "closure" to mass–energy (total isolation) is violated. In this case, conservation of invariant mass of the system also will no longer hold. Such a loss of rest mass in systems when energy is removed, according to E = mc2 where E is the energy removed, and m is the change in rest mass, reflect changes of mass associated with movement of energy, not "conversion" of mass to energy.
Again, in special relativity, the rest mass of a system is not required to be equal to the sum of the rest masses of the parts (a situation which would be analogous to gross mass-conservation in chemistry). For example, a massive particle can decay into photons which individually have no mass, but which (as a system) preserve the invariant mass of the particle which produced them. Also a box of moving non-interacting particles (e.g., photons, or an ideal gas) will have a larger invariant mass than the sum of the rest masses of the particles which compose it. This is because the total energy of all particles and fields in a system must be summed, and this quantity, as seen in the center of momentum frame, and divided by c2, is the system's invariant mass.
In special relativity, mass is not "converted" to energy, for all types of energy still retain their associated mass. Neither energy nor invariant mass can be destroyed in special relativity, and each is separately conserved over time in closed systems. Thus, a system's invariant mass may change only because invariant mass is allowed to escape, perhaps as light or heat. Thus, when reactions (whether chemical or nuclear) release energy in the form of heat and light, if the heat and light is not allowed to escape (the system is closed and isolated), the energy will continue to contribute to the system rest mass, and the system mass will not change. Only if the energy is released to the environment will the mass be lost; this is because the associated mass has been allowed out of the system, where it contributes to the mass of the surroundings. 
Concepts that were similar to what nowadays is called "relativistic mass", were already developed before the advent of special relativity. For example, it was recognized by J. J. Thomson in 1881 that a charged body is harder to set in motion than an uncharged body, which was worked out in more detail by Oliver Heaviside (1889) and George Frederick Charles Searle (1897). So the electrostatic energy behaves as having some sort of electromagnetic mass , which can increase the normal mechanical mass of the bodies.  
Then, it was pointed out by Thomson and Searle that this electromagnetic mass also increases with velocity. This was further elaborated by Hendrik Lorentz (1899, 1904) in the framework of Lorentz ether theory. He defined mass as the ratio of force to acceleration, not as the ratio of momentum to velocity, so he needed to distinguish between the mass parallel to the direction of motion and the mass perpendicular to the direction of motion (where is the Lorentz factor, v is the relative velocity between the aether and the object, and c is the speed of light). Only when the force is perpendicular to the velocity, Lorentz's mass is equal to what is now called "relativistic mass". Max Abraham (1902) called longitudinal mass and transverse mass (although Abraham used more complicated expressions than Lorentz's relativistic ones). So, according to Lorentz's theory no body can reach the speed of light because the mass becomes infinitely large at this velocity.   
Albert Einstein also initially used the concepts of longitudinal and transverse mass in his 1905 electrodynamics paper (equivalent to those of Lorentz, but with a different by an unfortunate force definition, which was later corrected), and in another paper in 1906.   However, he later abandoned velocity dependent mass concepts (see quote at the end of next section).
The precise relativistic expression (which is equivalent to Lorentz's) relating force and acceleration for a particle with non-zero rest mass moving in the x direction with velocity v and associated Lorentz factor is
In special relativity, an object that has nonzero rest mass cannot travel at the speed of light. As the object approaches the speed of light, the object's energy and momentum increase without bound.
In the first years after 1905, following Lorentz and Einstein, the terms longitudinal and transverse mass were still in use. However, those expressions were replaced by the concept of relativistic mass, an expression which was first defined by Gilbert N. Lewis and Richard C. Tolman in 1909.  They defined the total energy and mass of a body as
Tolman in 1912 further elaborated on this concept, and stated: "the expression m0(1 − v2/c2)−1/2 is best suited for THE mass of a moving body."   
In 1934, Tolman argued that the relativistic mass formula holds for all particles, including those moving at the speed of light, while the formula only applies to a slower-than-light particle (a particle with a nonzero rest mass). Tolman remarked on this relation that "We have, moreover, of course the experimental verification of the expression in the case of moving electrons ... We shall hence have no hesitation in accepting the expression as correct in general for the mass of a moving particle." 
When the relative velocity is zero, is simply equal to 1, and the relativistic mass is reduced to the rest mass as one can see in the next two equations below. As the velocity increases toward the speed of light c, the denominator of the right side approaches zero, and consequently approaches infinity. While Newton's second law remains valid in the form
Even though Einstein initially used the expressions "longitudinal" and "transverse" mass in two papers (see previous section), in his first paper on (1905) he treated m as what would now be called the rest mass.  Einstein never derived an equation for "relativistic mass", and in later years he expressed his dislike of the idea: 
It is not good to introduce the concept of the mass of a moving body for which no clear definition can be given. It is better to introduce no other mass concept than the ’rest mass’ m. Instead of introducing M it is better to mention the expression for the momentum and energy of a body in motion.— Albert Einstein in letter to Lincoln Barnett, 19 June 1948 (quote from L.B. Okun (1989), p. 42 )
The concept of relativistic mass is widely used in popular science writing and in high school and undergraduate textbooks. Authors such as Okun and A. B. Arons have argued against this as archaic and confusing, and not in accord with modern relativistic theory.   Arons wrote: 
For many years it was conventional to enter the discussion of dynamics through derivation of the relativistic mass, that is the mass–velocity relation, and this is probably still the dominant mode in textbooks. More recently, however, it has been increasingly recognized that relativistic mass is a troublesome and dubious concept. [See, for example, Okun (1989). ]... The sound and rigorous approach to relativistic dynamics is through direct development of that expression for momentum that ensures conservation of momentum in all frames:rather than through relativistic mass.
C. Alder takes a similarly dismissive stance on mass in relativity. Writing on said subject matter, he says that "its introduction into the theory of special relativity was much in the way of a historical accident", noting towards the widespread knowledge of E = mc2 and how the public's interpretation of the equation has largely informed how it is taught in higher education.  He instead supposes that the difference between rest and relativistic mass should be explicitly taught, so that students know why mass should be thought of as invariant "in most discussions of inertia".
Many contemporary authors such as Taylor and Wheeler avoid using the concept of relativistic mass altogether:
The concept of "relativistic mass" is subject to misunderstanding. That's why we don't use it. First, it applies the name mass – belonging to the magnitude of a 4-vector – to a very different concept, the time component of a 4-vector. Second, it makes increase of energy of an object with velocity or momentum appear to be connected with some change in internal structure of the object. In reality, the increase of energy with velocity originates not in the object but in the geometric properties of spacetime itself. 
While spacetime has the unbounded geometry of Minkowski space, the velocity-space is bounded by c and has the geometry of hyperbolic geometry where relativistic mass plays an analogous role to that of Newtonian mass in the barycentric coordinates of Euclidean geometry.  The connection of velocity to hyperbolic geometry enables the 3-velocity-dependent relativistic mass to be related to the 4-velocity Minkowski formalism.