The answers referring to markeredgewidth work. There's a new argument now that is more descpritive capthick for cap thickness. From the documentation of version 3.7.2 of matplotlib.
capthick : float, default: None An alias to the keyword argument *markeredgewidth* (a.k.a. *mew*). This setting is a more sensible name for the property that controls the thickness of the error bar cap in points. For backwards compatibility, if *mew* or *markeredgewidth* are given, then they will over-ride *capthick*. This may change in future releases.