| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | /* bitops.S: Sparc64 atomic bit operations. | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  |  * | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  |  * Copyright (C) 2000, 2007 David S. Miller (davem@davemloft.net)
 | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  |  */ | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #include <asm/asi.h> | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | #include <asm/backoff.h> | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	.text | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.globl	test_and_set_bit
 | 
					
						
							|  |  |  | 	.type	test_and_set_bit,#function | 
					
						
							|  |  |  | test_and_set_bit:	/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	or	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 and	%g7, %o2, %g2 | 
					
						
							|  |  |  | 	clr	%o0 | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	movrne	%g2, 1, %o0 | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	retl | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	test_and_set_bit, .-test_and_set_bit | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 	.globl	test_and_clear_bit
 | 
					
						
							|  |  |  | 	.type	test_and_clear_bit,#function | 
					
						
							|  |  |  | test_and_clear_bit:	/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	andn	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 and	%g7, %o2, %g2 | 
					
						
							|  |  |  | 	clr	%o0 | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	movrne	%g2, 1, %o0 | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	retl | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	test_and_clear_bit, .-test_and_clear_bit | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 	.globl	test_and_change_bit
 | 
					
						
							|  |  |  | 	.type	test_and_change_bit,#function | 
					
						
							|  |  |  | test_and_change_bit:	/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	xor	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 and	%g7, %o2, %g2 | 
					
						
							|  |  |  | 	clr	%o0 | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	movrne	%g2, 1, %o0 | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	retl | 
					
						
							| 
									
										
										
											
												[SPARC64]: Avoid membar instructions in delay slots.
In particular, avoid membar instructions in the delay
slot of a jmpl instruction.
UltraSPARC-I, II, IIi, and IIe have a bug, documented in
the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51
The long and short of it is that if the IMU unit misses
on a branch or jmpl, and there is a store buffer synchronizing
membar in the delay slot, the chip can stop fetching instructions.
If interrupts are enabled or some other trap is enabled, the
chip will unwedge itself, but performance will suffer.
We already had a workaround for this bug in a few spots, but
it's better to have the entire tree sanitized for this rule.
Signed-off-by: David S. Miller <davem@davemloft.net>
											
										 
											2005-06-27 15:42:04 -07:00
										 |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	test_and_change_bit, .-test_and_change_bit | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 	.globl	set_bit
 | 
					
						
							|  |  |  | 	.type	set_bit,#function | 
					
						
							|  |  |  | set_bit:		/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	or	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 nop | 
					
						
							|  |  |  | 	retl | 
					
						
							|  |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	set_bit, .-set_bit | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 	.globl	clear_bit
 | 
					
						
							|  |  |  | 	.type	clear_bit,#function | 
					
						
							|  |  |  | clear_bit:		/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	andn	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 nop | 
					
						
							|  |  |  | 	retl | 
					
						
							|  |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	clear_bit, .-clear_bit | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 	.globl	change_bit
 | 
					
						
							|  |  |  | 	.type	change_bit,#function | 
					
						
							|  |  |  | change_bit:		/* %o0=nr, %o1=addr */ | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	BACKOFF_SETUP(%o3) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	srlx	%o0, 6, %g1 | 
					
						
							|  |  |  | 	mov	1, %o2 | 
					
						
							|  |  |  | 	sllx	%g1, 3, %g3 | 
					
						
							|  |  |  | 	and	%o0, 63, %g2 | 
					
						
							|  |  |  | 	sllx	%o2, %g2, %o2 | 
					
						
							|  |  |  | 	add	%o1, %g3, %o1 | 
					
						
							|  |  |  | 1:	ldx	[%o1], %g7 | 
					
						
							|  |  |  | 	xor	%g7, %o2, %g1 | 
					
						
							|  |  |  | 	casx	[%o1], %g7, %g1 | 
					
						
							|  |  |  | 	cmp	%g7, %g1 | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 	bne,pn	%xcc, 2f | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	 nop | 
					
						
							|  |  |  | 	retl | 
					
						
							|  |  |  | 	 nop | 
					
						
							| 
									
										
										
										
											2007-10-15 16:41:44 -07:00
										 |  |  | 2:	BACKOFF_SPIN(%o3, %o4, 1b) | 
					
						
							| 
									
										
										
										
											2005-04-16 15:20:36 -07:00
										 |  |  | 	.size	change_bit, .-change_bit |