
652 F.3d 653 (2011)
Richard COLE, Plaintiff-Appellant,
v.
Michael ASTRUE, Commissioner of Social Security, Defendant-Appellee.
No. 09-4309.
United States Court of Appeals, Sixth Circuit.
Argued: December 8, 2010.
Decided and Filed: July 15, 2011.
Rehearing Granted September 21, 2011.
*655 ARGUED: Clifford M. Farrell, Manring & Farrell, Columbus, Ohio, for Appellant. Brian J. Saame, Social Security Administration, Chicago, Illinois, for Appellee. ON BRIEF: Clifford M. Farrell, Manring & Farrell, Columbus, Ohio, for Appellant. Brian J. Saame, Social Security Administration, Chicago, Illinois, for Appellee.
Before: MOORE and STRANCH, Circuit Judges; COHN, District Judge.[*]


*656 OPINION
JANE B. STRANCH, Circuit Judge.
Richard L. Cole suffered a work-related back injury, subsequently developed depression then sought disability. The Commissioner of Social Security adjudged him not to be disabled for purposes of social security disability benefits. Cole appeals from the district court's judgment affirming the Commissioner's decision. We reverse and remand.

I. Background

A. Factual Background
Cole was involved in a work-related vehicle accident that injured his back in 1994. He continued working off and on until June 2000, when pain prohibited his work as a truck driver. A November 1999 MRI revealed a mild disc bulge at the L4-5 level with spinal canal narrowing. By September 15, 2000, a MRI showed Cole's injury to have digressed to "a prominent diffuse disc bulge." Cole had back surgery on July 20, 2001. By August, Cole's pain and other symptoms had improved markedly, and in February 2002, Cole informed doctors he had experienced complete relief of pain down his legs.
Sometime thereafter, Cole began experiencing increasing pain in his back. His orthopedic spine surgeon, Dr. Daryl Sybert, diagnosed him as having chronic myofascial pain related to his surgical fusion. However, tests revealed no abnormalities and Dr. Sybert released Cole to go back to work with the restriction of no repetitive lifting greater than 50 lbs. In May 2002, Dr. Sybert opined that there was not much more he could do for Cole other than to monitor the fusion point.
Cole began to be absent from his scheduled appointments apparently because his worker's compensation provider ceased paying for these visits. He had a MRI in February 2004 which revealed only post-operative changes to his spine, but he failed to follow up with Dr. Sybert about the results until February 22, 2005. From March 2004 until March 2005, Cole saw Dr. Julie Rindler on several occasions for physical rehabilitation, but he stopped these visits, despite their benefit, due to insurance problems. Dr. Rindler also noted there was little more she could do for Cole. In February 2004, Cole's family physician, Dr. Richard Potts, opined that Cole was then unemployable but was improving with physical therapy and counseling.
Cole first sought mental health counseling in August 2002 due to difficulty sleeping, poor eating habits, weight loss of nearly 40 pounds in a two month period, and anxiety. His counselor, Melodee Bass, gave Cole a DSM-IV diagnosis of mood disorder with depressive features.
In September 2002, Ms. Bass referred Cole to a psychiatrist, Dr. Kalpana Vishnupad, to set up a medication regimen. He was initially assessed by social worker Mary Dailey, who diagnosed Cole as suffering major depression, single event, severe without psychotic features as per DSM-IV guidelines. Cole saw both regularly for counseling and therapy. On April 21, 2003, Dr. Vishnupad rated Cole's residual functional capacity ("RFC") due to his mental condition as "marked" impairment in the five categories of social interaction, "marked" and "moderate" impairment in the six categories of sustained concentration and persistence, and "marked" impairment in the six categories of adaptation to work conditions.[1] Dr. Vishnupad noted *657 that Cole "continues with agitation and irritability, loses temper frequently  has had `blow ups' on jobs in the past with other employees/supervisors." She further opined that Cole's condition would likely deteriorate if placed under the stress of a job, and that his impairments would not likely improve even if only minimal contact or interaction with others was required.
Cole saw Ms. Dailey and Dr. Vishnupad regularly until October 2004, when he failed to appear for a scheduled counseling appointment and his case was terminated. However, Cole reappeared for counseling in August 2005 and his case was reopened. Two months later, he again began missing appointments, and Ms. Dailey terminated his case. However, Cole continued to see Dr. Vishnupad periodically for medication management. Throughout 2006, Dr. Vishnupad most often noted Cole's mood as "fair" or "ok," adjusting his medications accordingly.
During the course of therapy, Cole saw two consulting psychologists. He saw Dr. Michael Farrell, an employer-selected consultant, in June 2003 in conjunction with his worker's compensation claims. During the consultation, Dr. Farrell conducted a Minnesota Multiphasic Personality Inventory-2 (MMPI-2), but declared the results invalid due to Cole's perceived over-reporting of symptoms. He concluded that Cole did not meet the DSM-IV criteria for major depressive disorder, and instead showed signs of chronic and low grade depression.
On March 23, 2004, Cole saw Dr. Owen Ward, a psychologist who gave a consultative opinion of Cole's mental impairments. Dr. Ward stated Dr. Farrell's opinion was flatly incorrect due to Dr. Farrell's misreading of the DSM-IV criteria. Dr. Ward agreed with Dr. Vishnupad's diagnosis of major depressive disorder, single episode, without psychotic features.
Cole alleged disability due to his back problems during a closed period from June 2000 to February 2002 and since August 2002 due to his combined physical and mental impairments.

B. Procedural Background
In September 2001, Mr. Cole filed for social security disability insurance benefits and was denied. His first full disability hearing was conducted on July 26, 2004. The ALJ heard testimony from Cole, the Commission's medical expert ("ME") Arthur Lorver, and the Commission's vocational expert ("VE") Vanessa Harris.
Lorver testified that, after surgery, Cole did not meet or equal Listing 1.04,[2] although he would require a sedentary job in order to be employable. Lorver testified only as to Cole's physical impairments, and the Commissioner offered no expert testimony on his mental impairments. Harris testified that there were approximately 3,200 jobs in the region that Cole could perform given the restrictions imposed by the ALJ. Taking Dr. Vishnupad's RFC of his mental impairments as "marked," Ms. Harris testified that none of the 3,200 jobs would be available.
On September 21, 2004, the ALJ issued a decision finding that Cole was not disabled due to his ability to perform a significant number of jobs in the economy. In coming to that conclusion, the ALJ declined to give Dr. Vishnupad's RFC assessment controlling weight due to its perceived conflict with other evidence on the record, namely Cole's own testimony about his daily activities.
*658 Cole challenged the decision in federal court. The Magistrate Judge opined that the ALJ's decision was not supported by substantial evidence, finding the ALJ failed to articulate any reasons for partially rejecting the testimony of the Commission's medical expert. On June 8, 2006, the district court adopted the Magistrate's Report and Recommendations and remanded the matter for re-evaluation of the record.
A second full hearing was held before the ALJ on January 24, 2007, at which Cole, the Commission's ME Richard Hutson, and the Commission's VE Charlotta Ewers testified. Hutson testified that Cole did not meet or equal Listing 1.04 because of his physical impairments. The Commissioner did not offer expert testimony as to his mental impairments. Ewers testified that, given the job restrictions imposed by the ALJ, there were 4,000 suitable jobs in the region. Taking Dr. Vishnupad's RFC as true, Ewers said that Cole would not be capable of doing the jobs identified.
The ALJ issued a decision on June 8, 2007, again finding Cole was not disabled. While the ALJ found Cole's physical and mental impairments to be severe, he concluded that they did not meet or equal the appropriate listing level and that Cole's RFC allowed him to perform a significant number of jobs in the economy. The ALJ determined that the opinion of Cole's treating physician, Dr. Potts, that Cole was unemployable did not merit controlling weight because it was not well supported by objective evidence and not consistent with other substantial evidence. Physically, the ALJ found that Cole had a mild limitation in activities of daily living, noting that Cole "helped out with laundry, cooked dinner, drove regularly, and attended church services and softball games." Mentally, the ALJ found that Cole has a moderate limitation in social functioning, based on his relatively successful work history with no "significant interpersonal conflicts with anyone at work." He also found Cole to have a moderate limitation in concentration, persistence or pace due to his chronic depression.
Ultimately, the ALJ concluded Cole could return to sedentary work with a sit/stand option and a number of other physical and stress-related restrictions and adopted Ewers's testimony about available jobs. Based on the applicable grid regulation,[3] the ALJ concluded Cole was not disabled.
Cole sought review before the agency Appeals Council, which declined jurisdiction. Cole then filed a federal suit, and the Magistrate Judge recommended affirming the agency's decision. On September 4, 2009, the district court affirmed the ALJ's decision. This appeal timely followed.

II. Analysis

A. Standard of Review
This Court reviews district court decisions regarding social security benefits determinations de novo. Ealy v. Comm'r of Soc. Sec., 594 F.3d 504, 512 (6th Cir. 2010). However, that review "is limited to determining whether the Commissioner's decision is supported by substantial evidence and was made pursuant to proper legal standards." Id. (quoting Rogers v. Comm'r of Soc. Sec., 486 F.3d 234, 241 (6th Cir.2007)). The substantial-evidence standard requires the Court to affirm the Commissioner's findings if they are supported by "such relevant evidence as a reasonable mind might accept as adequate *659 to support a conclusion." Richardson v. Perales, 402 U.S. 389, 401, 91 S.Ct. 1420, 28 L.Ed.2d 842 (1971) (citation omitted). An ALJ's failure to follow agency rules and regulations "denotes a lack of substantial evidence, even where the conclusion of the ALJ may be justified based upon the record." Blakley v. Comm'r of Soc. Sec., 581 F.3d 399, 407 (6th Cir.2009) (citations omitted).

B. The Treating Physician Rule and the Good Reasons Requirement
The Commissioner has elected to impose certain standards on the treatment of medical source evidence. 20 C.F.R. § 404.1502. Under one such standard, commonly called the treating physician rule, the Commissioner has mandated that the ALJ "will" give a treating source's opinion controlling weight if it "is well-supported by medically acceptable clinical and laboratory diagnostic techniques and is not inconsistent with the other substantial evidence in your case record." 20 C.F.R. § 404.1527(d)(2). If the ALJ declines to give a treating source's opinion controlling weight, he must then balance the following factors to determine what weight to give it: "the length of the treatment relationship and the frequency of examination, the nature and extent of the treatment relationship, supportability of the opinion, consistency of the opinion with the record as a whole, and specialization of the treating source." Wilson v. Comm'r of Soc. Sec., 378 F.3d 541, 544 (6th Cir.2004) (citing 20 C.F.R. § 404.1527(d)(2)).
Importantly, the Commissioner imposes on its decision makers a clear duty to "always give good reasons in our notice of determination or decision for the weight we give [a] treating source's opinion." 20 C.F.R. § 404.1527(d)(2). Those good reasons must be "supported by the evidence in the case record, and must be sufficiently specific to make clear to any subsequent reviewers the weight the adjudicator gave to the treating source's medical opinion and the reasons for that weight." Soc. Sec. Rul. No. 96-2p, 1996 SSR LEXIS 9, at *12 (Soc. Sec. Admin. July 2, 1996). This requirement is not simply a formality; it is to safeguard the claimant's procedural rights. It is intended "to let claimants understand the disposition of their cases, particularly in situations where a claimant knows that his physician has deemed him disabled and therefore might be especially bewildered when told by an administrative bureaucracy that []he is not." Wilson, 378 F.3d at 544. Significantly, the requirement safeguards a reviewing court's time, as it "permits meaningful" and efficient "review of the ALJ's application of the [treating physician] rule." Id. at 544-45.
At issue in this case is opinion evidence from Cole's four mental health sources, Drs. Vishnupad, Ward, Farrell, and counselor Dailey. To judge compliance with the treating source rule, we first determine, de novo, if any of those four are treating sources. See Smith v. Comm'r of Soc. Sec., 482 F.3d 873, 876 (6th Cir.2007). Both Dr. Vishnupad and Ms. Dailey are treating sources because both "[have], or [have] had, an ongoing treatment relationship" with Cole. 20 C.F.R. § 404.1502. However, as to both Vishnupad and Dailey, the ALJ failed to properly apply the treating physician rule and the good reasons rule.
As to Dr. Vishnupad, the ALJ labeled her Cole's "treating psychiatrist" and accepted her diagnosis of major depression, finding it consistent with the medical record in light of the overall evidence. It appears the ALJ deemed Dr. Vishnupad's medical opinion to be deserving of controlling weight pursuant to § 1527(d)(2) ("not inconsistent with the other substantial evidence in [the] case *660 record"), although he fails to address weight explicitly.
After accepting Dr. Vishnupad's diagnosis, the ALJ then rejected the conclusions contained in her RFC assessment about the severity of Cole's impairments as they relate to work. Before doing so, the ALJ failed to conduct the balancing of factors to determine what weight should be accorded these treating source opinions, and the Commissioner conceded at oral argument that the ALJ did not assign a specific weight to Dr. Vishnupad's RFC assessment. This alone constitutes error, as "[a] finding that a treating source medical opinion ... is not entitled to controlling weight [does] not [mean] that the opinion should be rejected." Blakley v. Comm'r of Soc. Sec., 581 F.3d 399, 408 (6th Cir.2009).
The Commissioner argues that, even though the ALJ did not specifically address Dr. Vishnupad's 2003 RFC assessment in his 2007 decision, he correctly addressed it in the first decision and is not required to re-analyze it in a subsequent decision. While that concept is accurate, it was not satisfied here. In the first decision, the ALJ relied on Cole's testimony about his daily activities to decline to give controlling weight to Dr. Vishnupad's RFC assessment; but, he failed to conduct any subsequent analysis as to what weight to give the RFC. Balancing the Wilson factors is required to satisfy the second prong of the treating physician rule. The 2007 decision thus fails to complete the required analysis. It is also inadequate because the ALJ's 2007 factual basis for declining to give controlling weight to Dr. Vishnupad's opinion, Cole's testimony about his daily activities, changed between the first hearing and the second hearing. Thus, the 2007 decision regarding Dr. Vishnupad does not provide an adequate basis for the current decision.
In addition to balancing the factors to determine what weight to give a treating source opinion denied controlling weight, the agency specifically requires the ALJ to give good reasons for the weight actually assigned. This decision failed to assign a weight and failed to provide any reasons for disregarding Dr. Vishnupad's RFC assessment after having found her diagnosis deserving of controlling weight. Instead, the ALJ conducted his own analysis of Cole's RFC based on Cole's testimony. However, the ALJ's restatement of Cole's daily activities is contrary to Cole's testimony in the second hearing in a number of relevant respects. For example, Cole testified he never goes to church, but the ALJ stated he attended church regularly. The ALJ relied heavily on his finding that Cole had never had interpersonal conflicts at work, despite his mental impairments. However, Dr. Vishnupad wrote in her notes that Cole has had past "blow ups" at his place of employment. As this Court has recently held, the ALJ's focus on the claimant's ability to do certain activities in discounting the treating source's opinion does not constitute "good reasons" for doing so when the claimant's testimony and other record evidence contradict the ALJ's finding. See Johnson v. Comm'r Soc. Sec., 652 F.3d 646, 652 (6th Cir.2011) ("The ALJ's focus on claimant's ability to do `yard work' is misplaced; in his testimony he stated, `I don't dig in the garden or anything like that, nothing heavy or manual.'"). In addition, the ALJ fails to mention Ms. Dailey, Cole's treating counselor, in the analysis of Cole's diagnosis and RFC and gives no reasons for not crediting her opinions.
The assessment of the medical evidence conducted at a hearing is particularly important at Step 5 of the evaluation of a social security disability claim because the RFC articulated by the ALJ will be used by the vocational expert to assess the claimant's ability to perform work. In order *661 for a VE's testimony to constitute substantial evidence that a significant number of jobs exists, "the question[s] must accurately portray a claimant's physical and mental impairments." Ealy, 594 F.3d at 516.
While it may be true that the opinions of Dr. Vishnupad and Ms. Dailey, though categorized as treating source opinions, should not ultimately be accorded controlling weight as to Cole's RFC, the ALJ did not go through the required analysis to arrive at that conclusion. This Court has made clear that "[w]e do not hesitate to remand when the Commissioner has not provided `good reasons' for the weight given to a treating physician's opinion and we will continue remanding when we encounter opinions from ALJ's that do not comprehensively set forth the reasons for the weight assigned to a treating physician's opinion." Hensley v. Astrue, 573 F.3d 263, 267 (6th Cir.2009) (quoting Wilson, 378 F.3d at 545).[4]

C. Substantial Evidence
This leads us to the ultimate question: Is the Commissioner's decision supported by substantial evidence and was it made pursuant to proper legal standards? The answer is no. The ALJ's failure to follow agency rules and regulations "denotes a lack of substantial evidence, even where the conclusion of the ALJ may be justified based upon the record." Blakley, 581 F.3d at 407.
However, a violation of the good reasons rule can be deemed to be "harmless error" if "(1) a treating source's opinion is so patently deficient that the Commissioner could not possibly credit it; (2) if the Commissioner adopts the opinion of the treating source or makes findings consistent with the opinion; or (3) where the Commissioner has met the goal of § 1527(d)(2) ... even though she has not complied with the terms of the regulation." Friend v. Comm'r of Soc. Sec., 375 Fed. Appx. 543, 551 (6th Cir.2010) (quoting Wilson, 378 F.3d at 547). In this case, we do not find the ALJ's violation of the good reasons rule to be harmless. It cannot be said that Dr. Vishnupad's opinions are patently deficient, because the ALJ found them sufficient as to Cole's category of diagnosis. Likewise, the Commissioner did not adopt Dr. Vishnupad's RFC assessment or make findings consistent with it. Nor can we say that the goal of § 1527(d)(2) was met, because the ALJ's decision leaves this Court without a clear understanding of why the ALJ credited Dr. Vishnupad's opinion as to diagnosis but discredited it as to work limitations. The same is true of Ms. Dailey's opinions.
Our finding that the ALJ's decision is not supported by substantial evidence is based on the ALJ's violation of the agency's procedural rules. It may be true that, on remand, the Commissioner reaches the same conclusion as to Cole's disability while complying with the treating physician rule and the good reasons requirement; however, Cole will then be able to understand the Commissioner's rationale and the procedure through which the decision was reached. The case must be remanded. "To hold otherwise ... would afford the Commissioner the ability [to] violate the regulation[s] with impunity and render the protections promised therein illusory." Wilson, 378 F.3d at 546. Those protections must be afforded to Cole in the disposition of his claim.

III. Conclusion
For the foregoing reasons, we REVERSE the judgment of the district court *662 with instructions to REMAND to the Commissioner for further proceedings consistent with this opinion.
NOTES
[*]  The Honorable Avern Cohn, United States District Judge for the Eastern District of Michigan, sitting by designation.
[1]  "Marked impairments" are defined as those rendering the person "unable to function in this area from 26% to 50% of the work day or work week." "Moderate impairments" are defined as those rendering the person "unable to function in this area from 11% to 25% of the work day or work week."
[2]  20 C.F.R., Part 404, Subpart P, Appendix 1 (Listing of Impairments by the Commissioner).
[3]  Vocational Rule 201.28 20, C.F.R., Part 404, Subpart P, Appendix 2 (Medical-Vocational Guidelines promulgated by the Commissioner).
[4]  Thus, we need not address Mr. Cole's second assignment of error on appeal that the examining source opinions do not, in fact, contradict the treating source opinions.
